Investigate Server Logs with Linux

Use core GNU/Unix commands to filter, analyze, and report on server log data.

Introduction

30 Second Summary

Every web server quietly records who visits, what they request, and whether each request succeeded or failed. When errors spike or suspicious patterns appear, those log files hold the answers.

In this project, you will play the role of a junior sysadmin who just received a server access log full of suspicious activity. Using nothing but the Linux command line, you will filter errors, rank IP addresses by request count, and build reusable analysis pipelines to summarize your findings.

What You'll Build

You run a single command chain in the terminal and instantly see which IP is hammering the login page, how many 404 errors are in the log, and whether someone is attempting a brute-force attack.

By the end of this project, you'll have:

A search and filter workflow that pulls out lines matching specific error codes, IP addresses, or request types from a 30-line server access log.
A ranking pipeline that extracts fields from structured log data, counts patterns, and sorts them to reveal the top offending IP addresses in a single command chain.
An organized investigation directory with saved report files, wildcard-managed filenames, and environment variables that keep your commands short and reusable.
Secret Mission: A reusable script that automates the entire investigation and generates a formatted, timestamped report with one command.

Are there any prerequisites?

This project assumes you already have Ubuntu installed via Parallels on your Mac. If you have not set that up yet, complete the Ubuntu installation before starting this project.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your Investigation Lab

Every investigation starts with a workspace. Before you can hunt for suspicious activity in a server log, you need a project directory and some realistic data to practice on.

In this step, you will open the Bash terminal, verify your environment, create a project folder, and generate a 30-line sample access log. By the end, you will have real log data ready for analysis.

In this step, get ready to:

Verify your terminal environment and explore help commands.
Create a project directory and generate sample log data.
View raw log data with basic file viewing commands.

Verify your environment

Before running any investigation commands, you need to confirm your terminal is working and you know where you are in the filesystem.

In your Ubuntu VM, press Ctrl+Alt+T to open the Terminal application.

A terminal window appears with a blinking cursor. This is where you will run every command in this project.

Check your current location and system info by running these commands one at a time:

pwd

uname -a

echo $SHELL

You should see three pieces of information printed to the terminal.

What should I see?

pwd prints your current working directory. It should show something like /home/your-username.
uname -a displays your kernel version, architecture, and OS info.
echo $SHELL confirms you are running Bash. It should print /bin/bash.

Not seeing /bin/bash?

If echo $SHELL shows a different shell (like /bin/zsh), you can switch to Bash for this project by typing bash and pressing Enter.

help me switch my default shell to Bash on Ubuntu

Linux includes a built-in help system you can use whenever you encounter an unfamiliar command.

Open the manual page for uname by running:

man uname

What is a man page?

The man command opens the manual for any command installed on your system. It takes over the full terminal screen. Use the arrow keys to scroll, and press q to exit back to your prompt.

Press q to exit the man page.
Explore three more ways to look up commands by running these one at a time:

type cd

which ls

apropos directory

What do these help commands do?

type tells you whether a command is a shell builtin, an alias, or an external program.
which shows the full path to an external command's executable file.
apropos searches all man page descriptions for a keyword. Useful when you know what you want to do but not which command does it.

Create the project directory and generate log data

Your investigation needs a dedicated workspace. You will create a project directory and then generate a sample Apache-format access log inside it.

Create the project directory by running this command:

mkdir -p ~/log-investigation

Confirm the directory was created by running:

ls ~/log-investigation

The directory is empty for now. You should see no output, and no error message. That means the directory exists and is ready to go.

Next, you will create a script that generates a realistic 30-line server access log. You will use nano, a simple terminal-based text editor, to create the script file.

Open nano to create the script file by running:

nano ~/log-investigation/generate-logs.sh

What is nano?

Nano is a terminal-based text editor. It takes over the full terminal screen, similar to a man page. You will see a blank editing area with shortcut hints along the bottom. You can type and paste text just like a regular editor.

Copy the following code and paste it into the nano editor:

#!/bin/bash
LOG_FILE="$HOME/log-investigation/access.log"

mkdir -p "$HOME/log-investigation"

cat > "$LOG_FILE" << 'EOF'
192.168.1.10 - - [15/Aug/2026:08:12:33 +0000] "GET /index.html HTTP/1.1" 200 1043
10.0.0.5 - admin [15/Aug/2026:08:12:45 +0000] "POST /login HTTP/1.1" 200 532
192.168.1.10 - - [15/Aug/2026:08:13:01 +0000] "GET /about.html HTTP/1.1" 200 2048
172.16.0.22 - - [15/Aug/2026:08:13:15 +0000] "GET /products HTTP/1.1" 301 0
10.0.0.5 - admin [15/Aug/2026:08:14:02 +0000] "GET /dashboard HTTP/1.1" 200 4096
192.168.1.15 - - [15/Aug/2026:08:14:33 +0000] "GET /nonexistent HTTP/1.1" 404 123
172.16.0.22 - - [15/Aug/2026:08:15:01 +0000] "GET /api/data HTTP/1.1" 500 0
10.0.0.5 - admin [15/Aug/2026:08:15:22 +0000] "DELETE /users/5 HTTP/1.1" 403 0
192.168.1.10 - - [15/Aug/2026:08:16:05 +0000] "GET /style.css HTTP/1.1" 200 876
192.168.1.15 - - [15/Aug/2026:08:16:33 +0000] "GET /missing-page HTTP/1.1" 404 123
172.16.0.22 - - [15/Aug/2026:08:17:01 +0000] "GET /api/data HTTP/1.1" 500 0
10.0.0.99 - - [15/Aug/2026:08:17:15 +0000] "GET /admin HTTP/1.1" 403 0
10.0.0.99 - - [15/Aug/2026:08:17:30 +0000] "GET /admin/config HTTP/1.1" 403 0
10.0.0.99 - - [15/Aug/2026:08:17:45 +0000] "GET /admin/users HTTP/1.1" 403 0
192.168.1.10 - - [15/Aug/2026:08:18:02 +0000] "GET /contact.html HTTP/1.1" 200 1567

Your cursor should be at the end of the last line you pasted. Copy and paste the remaining lines directly below:

172.16.0.22 - - [15/Aug/2026:08:18:33 +0000] "POST /api/submit HTTP/1.1" 500 0
10.0.0.5 - admin [15/Aug/2026:08:19:01 +0000] "GET /reports HTTP/1.1" 200 3456
192.168.1.15 - - [15/Aug/2026:08:19:22 +0000] "GET /old-page HTTP/1.1" 404 123
192.168.1.10 - - [15/Aug/2026:08:20:05 +0000] "GET /images/logo.png HTTP/1.1" 200 15234
172.16.0.22 - - [15/Aug/2026:08:20:33 +0000] "GET /api/health HTTP/1.1" 200 45
10.0.0.99 - - [15/Aug/2026:08:21:01 +0000] "POST /login HTTP/1.1" 401 0
10.0.0.99 - - [15/Aug/2026:08:21:15 +0000] "POST /login HTTP/1.1" 401 0
10.0.0.99 - - [15/Aug/2026:08:21:30 +0000] "POST /login HTTP/1.1" 401 0
192.168.1.10 - - [15/Aug/2026:08:22:02 +0000] "GET /blog HTTP/1.1" 200 5678
10.0.0.5 - admin [15/Aug/2026:08:22:45 +0000] "PUT /settings HTTP/1.1" 200 128
172.16.0.22 - - [15/Aug/2026:08:23:01 +0000] "GET /api/data HTTP/1.1" 500 0
192.168.1.15 - - [15/Aug/2026:08:23:33 +0000] "GET /typo-page HTTP/1.1" 404 123
192.168.1.10 - - [15/Aug/2026:08:24:05 +0000] "GET /services.html HTTP/1.1" 200 2345
10.0.0.5 - admin [15/Aug/2026:08:24:33 +0000] "GET /logs HTTP/1.1" 200 8901
172.16.0.22 - - [15/Aug/2026:08:25:01 +0000] "POST /api/submit HTTP/1.1" 500 0
EOF

echo "Sample log file created at $LOG_FILE"
echo "$(wc -l < "$LOG_FILE") lines generated."

Save the file by pressing Ctrl+O, then press Enter to confirm the filename.
Exit nano by pressing Ctrl+X.

What does this script do?

The script uses a heredoc (the << 'EOF' block) to write 30 lines of realistic Apache-format access log data into access.log. Each line simulates a web server request with an IP address, timestamp, HTTP method, URL, status code, and response size.

Run the script to generate the log file:

bash ~/log-investigation/generate-logs.sh

You should see two lines of output confirming the log file was created and that 30 lines were generated.

Move into your project directory and list the files by running:

cd ~/log-investigation
ls -lh

You should see access.log and generate-logs.sh listed with their file sizes and timestamps.

Only seeing generate-logs.sh?

If access.log is missing, the script may have an error. Re-open it with nano ~/log-investigation/generate-logs.sh and check that the first line is #!/bin/bash and that EOF appears on its own line after the last log entry. Save, exit, and run the script again.

help me debug my generate-logs.sh script

✔️ Awesome, I've got everything!

Great. Make sure both access.log and generate-logs.sh appear in your ls -lh output, and you are ready to move on.

ⓧ I'd like to double check the full code

Open generate-logs.sh in nano by running nano ~/log-investigation/generate-logs.sh. Scroll through and confirm these sections appear in order:

The shebang line (#!/bin/bash), the LOG_FILE variable, the mkdir -p command, and the cat > ... << 'EOF' line (from the first paste block).
30 log lines starting with 192.168.1.10 and ending with 172.16.0.22, split across both paste blocks.
EOF on its own line, followed by the two echo statements (from the second paste block).

Press Ctrl+X to exit nano when you are done checking.

Take your first look at the log

You have a log file, but you have not looked inside it yet. Linux has several commands for viewing file contents, each showing the data in a different way.

Dump the entire file to the terminal by running:

cat access.log

All 30 lines scroll past in the terminal. You can see IP addresses, timestamps, URLs, and status codes, but they go by fast.

View just the first 10 lines by running:

head access.log

View the last 5 lines by running:

tail -n 5 access.log

What do head and tail do?

head shows the first 10 lines of a file by default. tail shows the last 10. The -n flag lets you change how many lines to display.

Count the total number of lines in the file by running:

wc -l access.log

You should see 30 access.log confirming the file has 30 lines.

View the file with line numbers by running:

nl access.log

Each line now has a number on the left, making it easier to reference specific entries. You can see the first line is a GET /index.html request and the last is a POST /api/submit request.

Now try to answer this question just by looking at the output: which IP address is causing the most errors? You would have to read every single line, check the status code, and count by hand. That is exactly the problem the next step solves.

Before you move on, let's confirm everything is in place. What do you think wc -l will show if you give it the full path to your log file?

Verify your log file using its full path by running:

wc -l ~/log-investigation/access.log

You should see 30 followed by the file path. Your investigation lab is ready.

Your workspace is set up and your log data is ready. Next up, you will learn to search and filter this data so you can answer real questions about it without reading every line.

Search and Filter Log Data

You have a 30-line server log and you know how to peek at it with cat, head, and tail. But scrolling through it line by line cannot tell you which IPs are causing errors or how many 404s occurred.

You need filtering tools to extract patterns from the noise. In this step, you will use grep to search for text patterns, cut to pull out specific fields, and sort and uniq to count and rank what you find.

In this step, get ready to:

Search for error patterns and filter log entries with grep.
Extract specific fields like IP addresses and URL paths with cut.
Count and rank patterns to find the top offenders using sort, uniq, and wc.

Find patterns with grep

Right now, answering "how many 404 errors are in this log?" means reading every single line and counting by hand. grep solves this by searching a file for lines that match a text pattern and printing only those lines.

Search for all 404 (Not Found) errors by running this command:

grep "404" access.log

You should see 4 lines printed, all from IP 192.168.1.15. Each line contains a request for a page that does not exist on the server.

Search for all 500 (Server Error) entries by running:

grep "500" access.log

You should see 5 lines, all from IP 172.16.0.22. One IP is responsible for every single server error.

What does grep do?

grep scans every line in a file and prints only the lines that contain your search pattern. The name stands for "global regular expression print."

Without grep, you would have to read all 30 lines yourself. With it, you get instant answers.

grep also has flags that change how it searches. Try these three variations to see what each flag does.

Find all POST requests using a case-insensitive search by running:

grep -i "post" access.log

You should see 6 lines. The -i flag tells grep to ignore upper/lowercase differences, so "post" matches "POST" in the log.

Show all lines that are NOT successful 200 responses by running:

grep -v "200" access.log

You should see 17 lines. The -v flag inverts the match, printing every line that does NOT contain the pattern.

Count the number of 500 errors without printing the lines by running:

grep -c "500" access.log

You should see just the number 5. The -c flag outputs only the count of matching lines instead of the lines themselves.

What do these grep flags mean?

-i means case-insensitive. It treats uppercase and lowercase letters as the same.
-v means invert. It shows every line that does NOT match your pattern.
-c means count. It prints the number of matching lines instead of the lines themselves.

Not seeing any output from grep?

Check that you are in the ~/log-investigation/ directory. Run pwd to confirm.
Make sure you typed the filename exactly as access.log. A typo like acess.log will produce an error.
Double-check your search term matches what is in the log. Patterns are case-sensitive unless you use -i.

Still stuck? Ask for help.

Extract specific fields with cut

grep gives you entire lines, but sometimes you only need one piece of information from each line. The log file has a structured format where fields are separated by spaces and quotes. cut lets you slice out individual fields by specifying a delimiter and a field number.

Extract just the IP addresses from every log entry by running:

cut -d' ' -f1 access.log

You should see 30 IP addresses, one per line. The -d' ' flag sets the delimiter to a space, and -f1 picks the first field. Since the IP address is always the first thing on each line, this pulls it out cleanly.

Extract the request line (like GET /index.html HTTP/1.1) by using a double-quote delimiter:

cut -d'"' -f2 access.log

You should see 30 request lines. The -d'"' flag sets the delimiter to a double-quote character. The request line sits between the first and second double quotes in each log entry, so -f2 grabs it.

Chain two cut commands together to extract just the URL path by running:

cut -d'"' -f2 access.log | cut -d' ' -f2

You should see just the URL paths like /index.html, /login, and /about.html. The first cut extracts the request line, then the | pipe sends that output into a second cut that pulls out just the URL path.

What is a pipe?

The | character is called a pipe. It takes the output of the command on its left and feeds it as input to the command on its right.

Pipes let you chain simple commands into powerful combinations. You will use them heavily for the rest of this project.

Seeing unexpected output from cut?

Make sure your delimiter is wrapped in quotes. For a space delimiter, use -d' '. For a double-quote delimiter, use -d'"'.
Check the field number. Fields are numbered starting from 1, not 0. -f1 grabs the first field, -f2 grabs the second, and so on.

Need help with cut syntax?

Count and rank with sort, uniq, and wc

You can now extract IP addresses from the log, but a raw list of 30 IPs is not much better than raw log lines. You need to count how many times each IP appears and rank them from most to least.

Sort all IP addresses alphabetically by running:

cut -d' ' -f1 access.log | sort

You should see the same 30 IPs, but now grouped together. All the 10.0.0.5 entries appear together, all the 172.16.0.22 entries appear together, and so on.

Count how many times each IP appears by piping the sorted output into uniq -c:

cut -d' ' -f1 access.log | sort | uniq -c

You should see 5 lines, each with a count and an IP address. uniq -c collapses adjacent identical lines into one and prefixes each with a count.

Why does uniq need sorted input?

uniq only compares adjacent lines. If the same IP appears on lines 1, 5, and 12, uniq would not recognize them as duplicates. Sorting first groups identical values together so uniq can count them correctly.

Rank IPs by request count with the most active IP first by running:

cut -d' ' -f1 access.log | sort | uniq -c | sort -rn

You should see the IPs ranked from most to fewest requests. 172.16.0.22 and 192.168.1.10 are tied at 7 requests each, making them the most active IPs in the log.

What does sort -rn do?

-n tells sort to treat the first field as a number. Without it, "7" would sort before "10" because "7" comes after "1" alphabetically.
-r reverses the order so the highest count appears first.

Now combine everything into one pipeline to answer a more specific question: which missing pages are requested the most?

Find the most-requested missing pages by running:

grep "404" access.log | cut -d'"' -f2 | cut -d' ' -f2 | sort | uniq -c | sort -rn

What does this pipeline do?

This single command chains six operations together. First, grep "404" filters to only the 404 error lines. Then the two cut commands extract just the URL path from each line.

Finally, sort | uniq -c | sort -rn groups, counts, and ranks the paths. You should see four missing pages, each requested once.

Getting wrong counts from uniq?

Make sure sort comes before uniq in your pipeline. Without sorting first, uniq will miss duplicates that are not on adjacent lines.
Check that the | pipe characters are all present. A missing pipe will cause the second command to be treated as a filename argument.

Need help building a pipeline?

Before you run the next command, think about how many 404 errors you saw earlier. Can you predict the exact count?

Verify your prediction by counting the 404 errors with this command:

grep "404" access.log | wc -l

You should see 4, confirming there are four 404 errors in the log.

Now try scrolling up in your terminal to find the output from your very first grep "404" command. Notice how it has been pushed off the screen by everything you have run since then. All your investigation results vanish when the terminal scrolls. You have no way to save findings, organize them, or reference them later.

You can now filter, extract, and rank any pattern in the log with a single command chain. Next up, you will learn to save these results to files and organize them into a proper investigation directory structure.

Organize Your Investigation Files

In Step 2, you built commands that answer questions like "which IP makes the most requests?" in a single pipeline. Your filtering and counting skills are solid.

There is one problem: every result vanishes the moment your terminal scrolls past it. All your investigation work lives in memory, with no way to save, organize, or reference findings later. In this step, you will capture command output to files, organize those files into directories, and set up shortcuts to avoid retyping long paths.

In this step, get ready to:

Save filtered log results to files using output redirection.
Organize files into directories with cp, mv, and wildcard patterns.
Set up environment variables to simplify repeated paths.

Save results with output redirection

Right now, every command you run prints its output to the screen and then it is gone. Output redirection lets you capture that output and write it directly into a file instead.

Create a reports directory and save all 404 error lines to a file by running these commands:

mkdir reports
grep "404" access.log > reports/errors.txt

What do these commands do?

mkdir reports creates a new directory called reports inside your current log-investigation folder.
> is the redirect operator. Instead of printing grep's output to the terminal, it writes it into reports/errors.txt. If the file does not exist, > creates it automatically.

View the saved errors by running:

cat reports/errors.txt

You should see the four log lines containing 404 status codes. This time, those results are saved to a file and will not vanish.

The > operator overwrites a file every time it runs. To add more data without erasing what is already there, use >> (the append operator).

Append all 500 error lines to the same file by running:

grep "500" access.log >> reports/errors.txt

Check that both error types are now in the file by running:

cat reports/errors.txt

You should see the four 404 lines from before, followed by five 500 lines that were appended.

Only seeing 500 lines, not 404?

You may have accidentally used > instead of >> for the second command. The single > overwrites the file, erasing the 404 lines. Re-run both commands: first grep "404" access.log > reports/errors.txt, then grep "500" access.log >> reports/errors.txt.

Help me understand the difference between > and >> in Bash

Save the IP ranking you built in Step 2 to a file by running:

cut -d' ' -f1 access.log | sort | uniq -c | sort -rn > reports/top-ips.txt

Confirm the ranking was saved by running:

cat reports/top-ips.txt

You should see IP addresses ranked by request count, with the most frequent IP at the top. That pipeline output is now saved permanently.

Linux has three standard streams: stdin (input, file descriptor 0), stdout (output, file descriptor 1), and stderr (errors, file descriptor 2). So far you have been redirecting stdout. You can also redirect stderr on its own.

Redirect stderr to /dev/null (a special file that discards everything) by running:

grep "nonexistent_file" access.log 2> /dev/null

What does 2> mean?

2> redirects file descriptor 2 (stderr) specifically. The pattern "nonexistent_file" does not match any log lines, so you see no output. If grep produced any error messages, they would be silently discarded into /dev/null.

Redirect both stdout and stderr at the same time by running:

cat access.log > /dev/null 2>&1

What does 2>&1 mean?

2>&1 sends stderr (2) to the same place as stdout (1). Combined with > /dev/null, this discards all output. You would use this pattern in scripts where you want a command to run silently.

Manage files with cp, mv, mkdir, and wildcards

Your reports directory is a flat pile of files. A real investigation needs structure: error reports in one place, summaries in another, backups of key findings.

Create two subdirectories inside reports by running:

mkdir -p reports/errors reports/summaries

What does the -p flag do?

The -p flag tells mkdir to create parent directories as needed and not throw an error if they already exist. Without -p, running this command twice would give you an error.

Confirm the new subdirectories exist by running:

ls reports/

You should see errors, errors.txt, summaries, and top-ips.txt listed.

Back up the error report and move the IP ranking into the summaries folder by running these commands:

cp reports/errors.txt reports/errors/errors-backup.txt
mv reports/top-ips.txt reports/summaries/

What is the difference between cp and mv?

cp copies a file, leaving the original in place. mv moves a file, removing it from the original location. After these commands, errors.txt still exists in reports/ (with a copy in reports/errors/), but top-ips.txt is now only in reports/summaries/.

Verify the backup landed in the right place by running:

ls reports/errors/

You should see errors-backup.txt listed.

Confirm the IP ranking was moved by running:

ls reports/summaries/

You should see top-ips.txt in its new home.

When you have many files, typing each name individually gets tedious. Wildcards (also called globbing patterns) let you match groups of files with a single expression.

Create three note files at once by running:

touch reports/notes-1.txt reports/notes-2.txt reports/notes-3.txt

List just the note files using the * wildcard by running:

ls reports/notes-*.txt

You should see notes-1.txt, notes-2.txt, and notes-3.txt listed. The * matches zero or more characters in that position.

Remove notes-1.txt and notes-2.txt using a bracket wildcard by running:

rm reports/notes-[1-2].txt

What does [1-2] match?

The [] wildcard matches any single character inside the brackets. [1-2] matches either 1 or 2, so the command removes both files in one shot.

Confirm only notes-3.txt remains using the ? wildcard by running:

ls reports/notes-?.txt

You should see only notes-3.txt. The ? wildcard matches exactly one character, so it finds notes-3 but would not match a file like notes-10.

Three wildcards to remember

* matches zero or more characters.
? matches exactly one character.
[abc] or [a-z] matches any single character in the set or range.

Use environment variables to simplify paths

Typing ~/log-investigation over and over is tedious and error-prone. Environment variables let you store a value once and reuse it everywhere.

Create a local variable called LOGDIR and verify its value by running these commands:

LOGDIR=~/log-investigation
echo $LOGDIR

You should see /home/<your-username>/log-investigation printed. The shell expanded ~ to your full home directory path.

What is a local variable vs an environment variable?

A local variable (also called a shell variable) exists only in the current shell session. If you open a new terminal or start a child shell, that variable is not available.

An environment variable is one that has been exported, which makes it visible to any child process. You need to export a variable for scripts and subshells to see it.

Export LOGDIR so child shells can access it by running:

export LOGDIR

Use the variable in a command to list your reports by running:

ls $LOGDIR/reports/

You should see the same directory listing as ls reports/. The shell replaced $LOGDIR with its stored value before running the command.

Confirm LOGDIR appears in the full environment by running:

env | grep LOGDIR

You should see a line showing LOGDIR=/home/<your-username>/log-investigation. The env command lists all environment variables, and piping to grep filters for just the one you want.

Verify the exported variable is accessible in a child shell by running:

bash -c 'echo $LOGDIR'

You should see the same path printed. This proves that export made the variable available beyond your current shell session.

Seeing a blank line instead of the path?

If bash -c 'echo $LOGDIR' prints an empty line, you may have skipped the export LOGDIR command. Without exporting, the variable exists only in your current shell. Run export LOGDIR and try again.

Also make sure you used single quotes around 'echo $LOGDIR' in the command. Double quotes would cause your current shell to expand the variable before passing it to the child shell.

Help me understand why my exported variable is not showing up in a child shell

Before you run the next command, think about what your reports directory should contain right now. How many files and subdirectories do you expect to see?

Check your prediction by running:

ls ~/log-investigation/reports/

You should see errors (directory), errors.txt (file), notes-3.txt (file), and summaries (directory). The IP ranking is no longer here because you moved it into reports/summaries/.

Confirm top-ips.txt is in its organized location by running:

ls ~/log-investigation/reports/summaries/

You should see top-ips.txt listed. Your investigation findings are now saved, backed up, and organized into a clear directory structure.

Your files are organized and your paths are simplified. Next up, you will chain commands together with pipes to build multi-step analysis pipelines that answer complex questions in a single line.

Build Analysis Pipelines

In the last step, you saved your investigation results to files and organized them into directories. You even set up environment variables to avoid retyping long paths.

But each command still runs on its own, and you have to manually redirect every result. Pipes let you chain operations into a single analysis pipeline that flows data from one command to the next. Command substitution lets you embed live output directly into filenames and strings.

In this step, get ready to:

Chain commands with pipes into multi-step analysis pipelines.
Use tee to save results to a file and display them on screen at the same time.
Use command substitution to create dynamic filenames and embed command output inside other commands.

Chain commands with pipes

A pipe (|) connects the output of one command directly into the input of the next. Instead of running five commands and saving intermediate files between each one, you chain them into a single line that processes data left to right.

Find which IPs trigger the most server errors by running this pipeline:

cat access.log | grep "500" | cut -d' ' -f1 | sort | uniq -c | sort -rn

What does this pipeline do?

cat access.log feeds the entire log file into the pipeline.
grep "500" filters down to only the lines containing a 500 status code.
cut -d' ' -f1 extracts just the IP address from each matching line.
sort | uniq -c | sort -rn groups identical IPs, counts them, and ranks by frequency (highest first).

Every 500 error comes from a single IP. That one address is hammering your server's API endpoints.

Detect possible brute-force login attempts by running this pipeline:

grep "POST /login" access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn

You should see that 10.0.0.99 made 3 login attempts, while 10.0.0.5 made just 1. Three rapid-fire login POSTs from a single IP is a classic brute-force signature.

Find the top 5 status codes across all requests by running this pipeline:

cut -d'"' -f3 access.log | cut -d' ' -f2 | sort | uniq -c | sort -rn | head -5

What does this pipeline do?

The first cut uses a double-quote delimiter (-d'"') to grab the third field, which is the part after the request string. The second cut extracts just the status code number from that field.

You should see 200 as the most common status code (successful requests), with error codes like 500, 404, 403, and 401 trailing behind it.

Seeing unexpected output?

Make sure you are still inside the ~/log-investigation/ directory. Run pwd to check.
Check that each | pipe character has a command on both sides. A pipe with nothing after it causes an error.
If you see a > prompt, your quotes are unmatched. Press Ctrl+C to cancel and retype the command.

Why is my pipe command producing no output?

Use tee to save and display simultaneously

So far, you have two choices: display output on screen with a pipe, or redirect it to a file with >. The tee command does both at once. It writes to a file AND passes the data through to the next command in the pipeline.

Display all failed login attempts (401 errors) and save them to a file at the same time by running this command:

grep "401" access.log | tee reports/failed-logins.txt

You should see three lines printed on screen, all showing POST requests to /login from 10.0.0.99 with a 401 status. Those same lines are now saved in reports/failed-logins.txt.

Append the forbidden access attempts (403 errors) to the same file by running this command:

grep "403" access.log | tee -a reports/failed-logins.txt

What does the -a flag do?

Without -a, tee overwrites the file (just like >). The -a flag appends instead (just like >>). Now failed-logins.txt contains both the 401 and 403 entries.

Confirm that both sets of entries are in the file by running this command:

cat reports/failed-logins.txt

You should see the three 401 lines at the top, followed by the four 403 lines at the bottom.

Save the full IP ranking to a file but only display the top 3 on screen by running this command:

cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | tee reports/summaries/ip-ranking.txt | head -3

What does this pipeline do?

The tee sits in the middle of the pipeline. It saves the full ranked list (all IPs) to ip-ranking.txt, then passes the same data forward to head -3, which trims the display to just the top 3. The file gets everything. Your screen gets the summary.

Verify the full ranking was saved by running this command:

cat reports/summaries/ip-ranking.txt

You should see all five IPs listed with their request counts. The file has the complete data, even though your screen only showed the top 3.

File is empty or missing?

Make sure the reports/summaries/ directory exists. Run ls reports/summaries/ to check. If it is missing, create it with mkdir -p reports/summaries.
Check that tee comes before head in the pipeline. If head comes first, it trims the data before tee can save it.

Why is my tee command not saving output to the file?

Use command substitution for dynamic operations

Command substitution lets you embed the output of one command inside another command. You wrap the inner command in $(), and the shell replaces it with whatever that command prints.

Embed the current date inside a string by running this command:

echo "Report generated on $(date)"

You should see something like Report generated on Fri Aug 15 08:30:00 UTC 2026. The shell ran date first, then inserted its output into the echo string.

Embed the log file's line count inside a string by running this command:

echo "Total log entries: $(wc -l < access.log)"

You should see Total log entries: 30. The < redirects the file into wc -l as stdin, so the output is just the number without the filename.

Create a timestamped backup of your access log by running this command:

cp access.log "reports/summaries/investigation-$(date +%Y%m%d-%H%M%S).txt"

What does this command do?

The $(date +%Y%m%d-%H%M%S) part runs the date command with a custom format: year, month, day, dash, hours, minutes, seconds. The result is a filename like investigation-20260815-083045.txt that is unique to the exact second it was created.

Confirm the timestamped file exists by running this command:

ls reports/summaries/investigation-*.txt

You should see your new timestamped file listed alongside the other files in the summaries/ directory.

Store a command's output in a variable using command substitution by running this command:

ERRORS=$(grep -c "500" access.log)

Use the stored value by running this command:

echo "Server errors found: $ERRORS"

You should see Server errors found: 5. The grep -c flag counted matching lines and command substitution captured that number into the ERRORS variable.

Why use command substitution in a variable?

Storing a count in a variable means you can reuse it later without running grep again. In longer scripts, this avoids repeating expensive searches and keeps your results consistent across multiple references.

Seeing an empty variable?

Make sure there are no spaces around the = sign. ERRORS=$(grep -c "500" access.log) is correct. ERRORS = $(grep ...) will fail.
Check that you used $() (dollar sign and parentheses), not ${} (dollar sign and braces). Braces are for variable expansion, not command substitution.

Why is my Bash variable empty after using command substitution?

Before you run the final check, think about what you would expect to see inside a file that was copied from access.log using command substitution in the filename.

Verify your timestamped investigation file by running this command:

cat ~/log-investigation/reports/summaries/investigation-*.txt

You should see all 30 lines of your access log printed to the terminal. The wildcard * matches the timestamp portion of the filename, so you do not need to remember the exact date and time.

Secret mission

Automate the Investigation with a Log Summary Script

Real sysadmins do not type the same analysis commands every day. In this mission, you will combine everything you have learned into a reusable Bash script that generates a formatted log analysis report automatically. The script introduces three new commands: sed, tr, and xargs.

Clean Up Your Resources

This project created files and directories only in your home directory under ~/log-investigation/. No cloud resources or paid services were used, so there are no ongoing costs.

Decide whether to keep your resources for continued practice or delete them entirely.

Resources you used:

~/log-investigation/ directory containing access.log, generate-logs.sh, log-summary.sh, and the reports/ subdirectory tree with saved report files.
LOGDIR environment variable set in your current terminal session.

Keep everything running

No action needed. Choose this if you want to keep practicing Linux commands with the log data and your investigation files.

Your ~/log-investigation/ directory and all its contents stay on disk for you to revisit.
The LOGDIR environment variable only exists in your current terminal session. It will disappear when you close the terminal.
You can re-run bash log-summary.sh any time to generate new timestamped reports.

Pause - I'll come back to this later

This project has no running processes or servers to shut down. Your files sit quietly on disk and take up minimal space.

Close your terminal window. The LOGDIR environment variable will be cleared automatically when the session ends.
When you come back, open a new terminal and run cd ~/log-investigation to pick up where you left off.
If you want your LOGDIR variable back, re-run export LOGDIR=~/log-investigation in your new session.

Delete - I don't want to use this again

Remove all project files and clean up your environment completely.

Remove the LOGDIR environment variable from your current session by running:

unset LOGDIR

Delete the entire project directory and all its contents by running:

rm -rf ~/log-investigation/

Confirm the directory is gone by running:

ls ~/log-investigation/

You should see a message saying the directory cannot be found. That confirms everything has been removed.

Nice Work!

You just investigated a web server access log from start to finish using nothing but the Linux command line. You started with 30 lines of raw data and built up to multi-stage pipelines that filter, rank, and report on suspicious activity in a single command.

You've learned how to:

Filtered and analyzed server log data using grep, cut, sort, uniq, and wc to extract error patterns, rank IP addresses by request count, and identify brute-force login attempts.
Organized investigation files into a structured directory tree using mkdir, cp, mv, wildcard patterns, and environment variables to save findings and manage paths efficiently.
Built multi-stage analysis pipelines using pipes, output redirection, tee, and command substitution to chain filters into single commands, save results to files, and generate timestamped reports.
Secret Mission: Automated the entire log investigation with a reusable log-summary.sh Bash script using sed, tr, xargs, and advanced multi-command pipelines.

Ready to quiz yourself?