Shell Scripting Part 2: Prod-Ready Bash

Harden a disk monitor script with strict mode, traps, and structured logging.

Introduction

30 Second Summary

Scripts that "work on my laptop" have a way of failing silently at 3am when no one is watching. A typo in a variable name produces wrong output. A failed command gets swallowed. Temp files pile up after crashes.

In this project, you will build a disk space monitoring script that starts naive and evolves into a production-hardened tool. Along the way, you will learn the exact patterns that separate fragile scripts from ones safe to run unattended.

What You'll Build

You will run your finished script from the terminal, watch it scan your disk mount points, and see structured log lines with timestamps and severity levels stream to both your screen and a log file.

By the end of this project, you'll have:

  • A disk space monitor you can run from your terminal that checks mount points, warns when usage exceeds a configurable threshold, and exits with meaningful codes you can check from cron or a CI runner.
  • Trap-based cleanup that guarantees temp files are removed even when the script crashes or you press Ctrl+C mid-execution.
  • Structured log output with timestamps and severity levels (INFO/WARN/ERROR) written to both your terminal and a log file, so you can diagnose what went wrong after the fact.
  • Input validation that rejects bad arguments immediately with clear error messages instead of producing corrupt output silently.
  • Secret Mission: extend the script with subcommands and a dry-run mode for safe testing in production environments.

Are there any prerequisites?

This project is Part 2 of the Basic to Intermediate Shell Scripting series. It assumes you have completed Part 1 and are comfortable with variables, quoting, conditionals, loops, functions with arguments, and exit codes.

You will need a Debian-based Linux system with Bash 5.2+ and Git installed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. You're going to take a disk space monitoring script from a fragile "works on my laptop" version to a production-hardened tool that's safe to run unattended at 3am.

Set Up Your Scripting Environment

Production scripts need a safety net that catches mistakes before runtime. ShellCheck is a static analysis linter that reads your shell scripts and flags bugs, portability issues, and bad patterns without ever executing the code.

In this step, you will install ShellCheck, create a Git-tracked project directory, and confirm everything works with a minimal test script.

In this step, get ready to:
  • Verify your Bash version and install ShellCheck.
  • Create the project directory with Git version control.
  • Write and lint your first test script.
Verify your Bash version and install ShellCheck

Your system already has Bash, but you need to confirm the version is 5.2 or higher before writing scripts that depend on modern features.

  • Check your Bash version by running this command:
bash --version

You should see output starting with a version number like 5.2.15 or higher.

Seeing a version below 5.2?

This project requires Bash 5.2+. If you see 4.x or 3.x, you are likely on an older distribution. Upgrade your system or install a newer Bash before continuing.

help me upgrade Bash to 5.2 on my Linux system

Next, you need ShellCheck. It analyzes scripts for bugs at the source level, catching issues you would otherwise only discover at runtime.

  • Check whether ShellCheck is already on your system by running:
shellcheck --version

✔️ I see version 0.9.0 or higher

ShellCheck is ready. You should see output that includes a line like version: 0.9.0 or higher. Continue to the next section below to create your project directory.

ⓧ I see an older version

You have ShellCheck installed, but it is below version 0.9.0. Upgrade it to the version in your package repositories.

  • Upgrade ShellCheck by running:
sudo apt update && sudo apt install --only-upgrade -y shellcheck
  • Verify the upgrade by running shellcheck --version again and confirming the version is 0.9.0 or higher.

ⓧ Command not found

ShellCheck is not installed yet. Install it from your distribution's package repository.

  • Install ShellCheck by running:
sudo apt update && sudo apt install -y shellcheck
  • Verify the install succeeded by running shellcheck --version again. You should now see version 0.9.0 or higher.

What does shellcheck --version output look like?

You will see several lines including the tool name, version number, license, and website. The key line to check is the one starting with version: followed by a number like 0.9.0.

Now create your project directory and initialize it with Git so you can track changes as the script evolves.

  • Create the project directory and initialize Git by running these commands:
mkdir ~/prod-scripts && cd ~/prod-scripts && git init

You should see a message like Initialized empty Git repository in /home/.../prod-scripts/.git/.

Permission denied or directory already exists?

If you see a "Permission denied" error, check that your user has write access to the home directory. If the directory already exists from a previous attempt, remove it first with rm -rf ~/prod-scripts and try again.

help me create a project directory on my Linux system

Create the scripts directory and confirm ShellCheck

Your scripts will live in a dedicated scripts/ subdirectory to keep the project organized.

  • Make sure you are inside your project directory, then create the scripts/ subdirectory by running:
cd ~/prod-scripts
mkdir scripts
  • Confirm the directory was created by running:
ls

You should see scripts listed in the output.

  • Run ShellCheck one more time to confirm it is working properly:
shellcheck --version

You should see version: 0.9.0 or higher in the output. This confirms ShellCheck is ready to lint your scripts.

Create and test your first script

Before building the disk monitor, you will create a minimal script to confirm the entire workflow: write, execute, lint.

  • Create the test script scripts/hello.sh by running:
cat > scripts/hello.sh << 'EOF'
#!/usr/bin/env bash
echo "Environment ready"
EOF

What does this command do?

The cat > file << 'EOF' pattern is called a heredoc redirect. It writes everything between the two EOF markers into the specified file. The single quotes around 'EOF' prevent the shell from expanding any variables inside the content.

  • Make the script executable by running:
chmod +x scripts/hello.sh
  • Run the script to confirm it works:
./scripts/hello.sh

You should see Environment ready printed to the terminal.

Now for the real test. Before you run ShellCheck on this file, what do you think the output will be for a two-line script with a valid shebang and a simple echo?

  • Lint the script by running:
shellcheck scripts/hello.sh

You should see no output at all. ShellCheck prints nothing when a file is clean, and exits with code 0. Silence means success.

Seeing warnings from ShellCheck?

Make sure the first line of scripts/hello.sh is exactly #!/usr/bin/env bash with no leading spaces or blank lines above it. A missing or incorrect shebang is the most common cause of ShellCheck warnings on simple scripts.

help me fix ShellCheck warnings on my hello.sh script

✔️ Awesome, I've got everything!

Great. Double check you have saved the file and confirmed both commands ran cleanly.

ⓧ I'd like to double check the full code

Here is the complete content of scripts/hello.sh:

#!/usr/bin/env bash
echo "Environment ready"

Your environment is locked in. You have ShellCheck catching bugs at the source level, a Git-tracked project directory, and a proven write-execute-lint workflow. Next up, you will write the naive disk monitor using your Part 1 skills.

Write the Naive Disk Monitor

Your scripting environment is ready. Now it's time to write something real using the Part 1 skills you already have.

You'll create a disk space monitoring script that checks how full each mount point is and warns you when usage exceeds a threshold. This version works under normal conditions, but the next step will expose its hidden fragility.

In this step, get ready to:
  • Write a check_disk function that extracts disk usage with df, tail, awk, and tr.
  • Loop over mount points and print OK or WARNING based on a configurable threshold.
  • Run the script and confirm it produces a clean disk usage report.
Create the script with a check_disk function

The df command reports disk usage for each filesystem. Your function will run it on a specific mount point and extract just the percentage number through a pipeline of text-processing commands.

  • Open a new file called scripts/disk_monitor.sh in your text editor by running:
nano scripts/disk_monitor.sh
  • Add the following code to define the shebang, configurable threshold, and the check_disk function:
#!/usr/bin/env bash

# --- Configuration ---
THRESHOLD="${1:-80}"

# --- Functions ---
check_disk() {
    local mount_point="$1"
    local usage
    usage=$(df "$mount_point" | tail -1 | awk '{print $5}' | tr -d '%')

    if [[ "$usage" -gt "$THRESHOLD" ]]; then
        echo "$mount_point: ${usage}% [WARNING]"
    else
        echo "$mount_point: ${usage}% [OK]"
    fi
}

What does this code do?

  • THRESHOLD="${1:-80}" reads the first script argument. If none is provided, it defaults to 80.
  • check_disk takes a mount point path as its argument and extracts the usage percentage.
  • df "$mount_point" prints disk usage. tail -1 grabs the data row (skipping the header). awk '{print $5}' picks the percentage column. tr -d '%' strips the % symbol so you can do a numeric comparison.
  • The if block compares usage to the threshold and prints either [WARNING] or [OK].
Add the report header and main loop

The function is ready but nothing calls it yet. You need a main section that prints a report header and loops over the mount points you want to monitor.

  • Below the check_disk function (after the closing }), add the following code:
# --- Main ---
echo "Disk Usage Report - $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

for mount in / /tmp; do
    check_disk "$mount"
done

What does this code do?

  • The three echo lines print a report header with the current date and the active threshold value.
  • $(date) is command substitution. It runs date and inserts its output inline.
  • The for loop iterates over / (root filesystem) and /tmp, calling check_disk for each.
  • Save the file and exit nano by pressing Ctrl+O, then Enter, then Ctrl+X.
Make the script executable and test it
  • Make the script executable by running:
chmod +x scripts/disk_monitor.sh

Before you run it, what do you expect to see? The script checks / and /tmp against a threshold of 80%. Will your mount points be under or over that threshold?

  • Run the disk monitor with the default threshold by running:
./scripts/disk_monitor.sh

You should see a report header with today's date, the threshold at 80%, and one line for each mount point showing its usage percentage followed by [OK] or [WARNING].

  • Now test with a custom threshold of 50 by running:
./scripts/disk_monitor.sh 50

This time the threshold is 50%. If any mount point exceeds 50% usage, it shows [WARNING] instead of [OK].

Script not producing output?

  • Make sure you're in the ~/prod-scripts directory. Run pwd to check.
  • If you see Permission denied, confirm you ran chmod +x scripts/disk_monitor.sh.
  • If you see a syntax error, check that all quotes are matched and the if [[ ... ]] block uses double brackets with spaces inside.

help me debug my disk_monitor.sh script

✔️ Awesome, I've got everything!

Great. Double check you've saved scripts/disk_monitor.sh and both runs produced output with mount point percentages.

ⓧ I'd like to double check the full code

Here is the complete scripts/disk_monitor.sh for comparison:

#!/usr/bin/env bash

# --- Configuration ---
THRESHOLD="${1:-80}"

# --- Functions ---
check_disk() {
    local mount_point="$1"
    local usage
    usage=$(df "$mount_point" | tail -1 | awk '{print $5}' | tr -d '%')

    if [[ "$usage" -gt "$THRESHOLD" ]]; then
        echo "$mount_point: ${usage}% [WARNING]"
    else
        echo "$mount_point: ${usage}% [OK]"
    fi
}

# --- Main ---
echo "Disk Usage Report - $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

for mount in / /tmp; do
    check_disk "$mount"
done

Your disk monitor works. It checks real mount points, applies a threshold, and prints a clear report. In the next step, you'll discover three ways this script silently fails in production.

Watch It Fail Silently

Your disk monitor from the previous step works perfectly under normal conditions. That is exactly the problem.

Production scripts face unexpected inputs, missing resources, and typos that slip through code review. Your naive script has three hidden failure modes that will bite you at 3am when nobody is watching. In this step, you will break the script on purpose and witness each failure firsthand.

In this step, get ready to:
  • Observe a failed command that the script completely ignores.
  • Introduce a variable typo that silently produces wrong results.
  • Witness a pipeline that hides upstream failures from the exit code.
Add a nonexistent mount point to the loop

The first failure mode is ignored command errors. When df tries to check a mount point that does not exist, it fails. The question is: does your script notice?

  • In scripts/disk_monitor.sh, find the for loop near the bottom of the file:
for mount in / /tmp; do
    check_disk "$mount"
done
  • Add /nonexistent to the list of mount points, so the line now looks like this:
for mount in / /tmp /nonexistent; do
    check_disk "$mount"
done
  • Save scripts/disk_monitor.sh.

Before you run this, what do you think will happen? Will the script crash when it hits /nonexistent, or will it keep going? What exit code do you expect?

  • Run the script and check its exit code by running this command:
./scripts/disk_monitor.sh; echo "Exit code: $?"

You will see an error message from df about /nonexistent on stderr. But the last line reads Exit code: 0. The script reported success despite a command failing inside it.

Why is this dangerous?

In production, exit code 0 means "everything is fine." Monitoring tools, cron schedulers, and CI pipelines all check exit codes to decide whether to alert you.

A script that returns 0 when something actually failed is lying to your infrastructure. Nobody gets paged. The problem festers silently.

Introduce a variable typo that produces wrong results

The second failure mode is unset variables. In Bash, referencing a variable that does not exist silently expands to an empty string. No error, no warning.

  • In scripts/disk_monitor.sh, find this line inside the check_disk function:
    if [[ "$usage" -gt "$THRESHOLD" ]]; then
  • Change $THRESHOLD to $THRESHHOLD (note the extra H). The line should now look like this:
    if [[ "$usage" -gt "$THRESHHOLD" ]]; then
  • Save the file and run the script by running this command:
./scripts/disk_monitor.sh

Every mount point now shows [WARNING] regardless of actual disk usage. The script did not crash. It did not print any error. It silently gave you completely wrong answers.

What happened under the hood?

The variable $THRESHHOLD does not exist, so Bash expands it to an empty string. The comparison [[ "45" -gt "" ]] treats the empty string as zero. Since any real disk usage is greater than zero, every mount triggers a WARNING.

Imagine this running in production: your pager fires for every single mount point, every single night. Or worse, if the comparison were reversed, you would never get paged even when disks are actually full.

Add a pipeline that masks upstream failures

The third failure mode is masked pipeline errors. In a pipeline like cat file | wc -l, Bash only reports the exit code of the last command. If cat fails but wc succeeds, the pipeline reports success.

  • In scripts/disk_monitor.sh, find the # --- Main --- section and locate this line:
echo "Disk Usage Report - $(date)"
  • Add this line directly above it (before the Disk Usage Report line):
echo "Pipeline test: $(cat /nonexistent/file | wc -l)"
  • Save the file.

Think about what you expect here. The file /nonexistent/file does not exist, so cat will fail. Will that failure stop the script, or will wc -l paper over it?

  • Run the script and check the exit code by running this command:
./scripts/disk_monitor.sh; echo "Exit code: $?"

You will see cat print an error to stderr, but the pipeline outputs Pipeline test: 0. The script continues running with corrupt data (zero lines from a file that does not exist) and still exits with code 0.

Why are these three failures so dangerous together?

Each failure mode silently corrupts your script's behavior without any visible indication that something went wrong. The exit code stays 0. The output looks plausible. No error reaches your monitoring.

These are exactly what set -euo pipefail prevents. set -e catches ignored errors. set -u catches unset variables. set -o pipefail catches masked pipeline failures. You will add all three in the next step.

Script won't run at all?

If you see a permission denied error, make sure the file is still executable. Run chmod +x scripts/disk_monitor.sh and try again.

If you see a syntax error, double-check that you only changed the specific parts mentioned above. The quotes and brackets must remain intact.

Help me debug my disk_monitor.sh script that won't execute

✔️ Awesome, I've got everything!

Great. Your script is intentionally broken right now. Do not fix it yet. The next step introduces the exact flags that prevent all three failure modes.

ⓧ I'd like to double check the full code

Your scripts/disk_monitor.sh should look exactly like this (with all three intentional breakages in place):

#!/usr/bin/env bash

# --- Configuration ---
THRESHOLD="${1:-80}"

# --- Functions ---
check_disk() {
    local mount_point="$1"
    local usage
    usage=$(df "$mount_point" | tail -1 | awk '{print $5}' | tr -d '%')

    if [[ "$usage" -gt "$THRESHHOLD" ]]; then
        echo "$mount_point: ${usage}% [WARNING]"
    else
        echo "$mount_point: ${usage}% [OK]"
    fi
}

# --- Main ---
echo "Pipeline test: $(cat /nonexistent/file | wc -l)"
echo "Disk Usage Report - $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

for mount in / /tmp /nonexistent; do
    check_disk "$mount"
done

You have now experienced the three silent killers of production scripts. Your script lies about its exit code, produces wrong output from a typo, and hides upstream data corruption. In the next step, you will add three flags and a trap handler that eliminate all of these failures permanently.

Harden with Strict Mode and Trap Handlers

In the previous step, you watched your script lie to you three different ways. It reported success when commands failed. It produced wrong results from an unset variable. It let a broken pipeline slide by unnoticed.

Each of those failures maps to a single Bash flag that prevents it. In this step, you will add all three flags in one line, then build a trap handler that guarantees temp files get cleaned up no matter how the script exits.

In this step, get ready to:
  • Enable strict mode with set -euo pipefail and watch it catch the failures that were previously silent.
  • Add a trap-based cleanup function that removes temp files even on crashes or Ctrl+C.
  • Handle expected failures gracefully so strict mode does not kill the script on normal conditions.
Enable strict mode

The three flags you need are bundled into a single line: set -euo pipefail. Each flag eliminates one of the failures you witnessed in Step 3.

  • In scripts/disk_monitor.sh, find the shebang line at the very top:
#!/usr/bin/env bash
  • Add set -euo pipefail on a new line directly below it, so the top of your file looks like this:
#!/usr/bin/env bash
set -euo pipefail

Before running the script, you also need to remove the pipeline test line you added in Step 3.

  • Find this line in the Main section:
echo "Pipeline test: $(cat /nonexistent/file | wc -l)"
  • Delete that entire line.
  • Save scripts/disk_monitor.sh.

What do you think will happen when you run the script now? The variable typo from Step 3 is still there.

  • Run the script to find out:
./scripts/disk_monitor.sh

You see an error like THRESHHOLD: unbound variable and the script exits immediately. That is set -u in action. It refuses to continue when you reference a variable that was never set.

What just happened?

In Step 3, this typo silently treated the threshold as zero and produced wrong results. Now set -u catches it immediately and stops the script before any damage is done.

  • Fix the typo now. Find this line inside the check_disk function:
    if [[ "$usage" -gt "$THRESHHOLD" ]]; then
  • Replace $THRESHHOLD with $THRESHOLD (one H, not two), so the line reads:
    if [[ "$usage" -gt "$THRESHOLD" ]]; then
  • Save the file and run the script again:
./scripts/disk_monitor.sh; echo "Exit code: $?"

This time the script gets past the variable check, processes / and /tmp successfully, then hits /nonexistent. The df command fails, and set -e kills the script with a non-zero exit code.

What do the three flags do?

  • set -e (errexit): exits the script when any command returns non-zero, unless that command is inside an if test.
  • set -u (nounset): exits the script when you reference an undefined variable.
  • set -o pipefail: makes a pipeline return the exit code of the rightmost command that failed, not just the last command.

Script exits immediately with no output?

  • Make sure you deleted the pipeline test line. If it is still present, set -o pipefail catches the cat /nonexistent/file failure before the loop even starts.
  • Confirm the typo is fixed. An unbound variable error means $THRESHHOLD (the typo) is still somewhere in your file.

Why is my script exiting before it processes any mount points?

Add the cleanup trap and temp file management

Strict mode stops the script when something goes wrong. But stopping mid-execution can leave temp files behind. A trap handler fires automatically when the script exits, guaranteeing cleanup happens no matter what.

  • In scripts/disk_monitor.sh, find the Configuration section. Replace it with the following, which adds a TMPFILE variable and a cleanup function:
# --- Configuration ---
THRESHOLD="${1:-80}"
TMPFILE=""

# --- Cleanup ---
cleanup() {
    local exit_code=$?
    if [[ -n "${TMPFILE:-}" && -f "$TMPFILE" ]]; then
        rm -f "$TMPFILE"
    fi
    exit "$exit_code"
}

trap cleanup EXIT

What does this code do?

  • TMPFILE="" initializes the variable to empty so it exists before the trap can reference it.
  • local exit_code=$? captures the exit code of whatever command triggered the trap. Without this, the cleanup function's own commands would overwrite it.
  • ${TMPFILE:-} uses the :- syntax to default to empty if TMPFILE is unset, preventing set -u from killing the cleanup itself.
  • trap cleanup EXIT registers the function to run on any exit: normal completion, set -e abort, explicit exit N, or after a signal handler runs.

Next, you need to update check_disk to handle its own errors gracefully and write results to the temp file.

  • Replace the entire check_disk function (from check_disk() { to its closing }) with this version:
# --- Functions ---
check_disk() {
    local mount_point="$1"
    local usage

    if ! df "$mount_point" > /dev/null 2>&1; then
        echo "ERROR: Cannot access mount point: $mount_point" >&2
        return 1
    fi

    usage=$(df "$mount_point" | tail -1 | awk '{print $5}' | tr -d '%')

    if [[ "$usage" -gt "$THRESHOLD" ]]; then
        echo "$mount_point: ${usage}% [WARNING]"
        echo "$mount_point: ${usage}% [ALERT]" >> "$TMPFILE"
        return 1
    else
        echo "$mount_point: ${usage}% [OK]"
        echo "$mount_point: ${usage}% [OK]" >> "$TMPFILE"
        return 0
    fi
}

What changed in check_disk?

  • The if ! df ... > /dev/null 2>&1 guard catches inaccessible mount points before they blow up the script. It prints a clear error to stderr and returns 1.
  • Results now write to $TMPFILE as a structured record, so you have a log even if the terminal output scrolls away.
  • Explicit return 0 and return 1 give the caller a clear signal about whether the mount point is healthy.
  • Now update the Main section. Find the current main section starting at echo "Disk Usage Report and replace everything from there to the end of the file with:
# --- Main ---
TMPFILE=$(mktemp /tmp/disk_monitor.XXXXXX)
echo "Disk Usage Report - $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

for mount in / /tmp /nonexistent; do
    check_disk "$mount"
done

Why mktemp?

mktemp /tmp/disk_monitor.XXXXXX creates a file with a unique, unpredictable name (the XXXXXX gets replaced with random characters). This prevents symlink attacks where a malicious process pre-creates a predictable filename to hijack your writes.

  • Save scripts/disk_monitor.sh.
  • Run the script to test the cleanup trap:
./scripts/disk_monitor.sh; echo "Exit code: $?"

The script processes / and /tmp successfully, then hits /nonexistent. The check_disk function returns 1, set -e kills the script, and the exit code is non-zero.

  • Now verify the cleanup trap worked by checking for leftover temp files:
ls /tmp/disk_monitor.*

You should see a "No such file or directory" message. The EXIT trap fired during the crash and cleaned up the temp file automatically.

Still seeing a temp file in /tmp?

  • Check that trap cleanup EXIT appears after the cleanup function definition, not before it. Bash must see the function before it can register it as a trap handler.
  • Make sure TMPFILE="" is in the Configuration section at the top. If it does not exist before the trap fires, the cleanup cannot find it.

Why is my trap not cleaning up the temp file?

Handle expected failures without killing the script

Right now, set -e kills the script when check_disk returns 1 for any reason. But a mount point exceeding its threshold is not a crash. It is expected behavior that the script should report, not die from.

  • Remove /nonexistent from the for loop (it was only there for demonstration).
  • Replace the entire Main section (from # --- Main --- to the end of the file) with this version:
# --- Main ---
TMPFILE=$(mktemp /tmp/disk_monitor.XXXXXX)
echo "Disk Usage Report - $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

alert_count=0
for mount in / /tmp; do
    if ! check_disk "$mount"; then
        ((alert_count++)) || true
    fi
done

if [[ "$alert_count" -gt 0 ]]; then
    exit 1
fi

What does this code do?

  • if ! check_disk "$mount" wraps the call in a conditional. Bash does not trigger set -e for commands inside an if test, so a non-zero return is handled instead of fatal.
  • ((alert_count++)) || true increments the counter. The || true is necessary because when alert_count increments from 0 to 1, the arithmetic expression evaluates to 0, which Bash treats as false. Without || true, set -e would kill the script on the first alert.
  • The final exit 1 only fires when alerts exist, giving the caller (cron, CI, another script) a meaningful exit code.
  • Save scripts/disk_monitor.sh.
  • Run shellcheck to verify your script has zero warnings:
shellcheck scripts/disk_monitor.sh

You should see no output, meaning zero warnings. A clean ShellCheck pass confirms your script follows best practices.

ShellCheck showing warnings?

  • If you see SC2086 (double quote to prevent globbing), make sure all variable references are quoted: "$TMPFILE", "$mount_point", "$THRESHOLD".
  • If you see SC2155 (declare and assign separately), check that local usage and the usage=$(...) assignment are on separate lines.

How do I fix ShellCheck warnings in my disk_monitor.sh script?

Now run the script itself to confirm everything works end to end. Before you do, predict: will the exit code be 0 or 1?

  • Run the final version:
./scripts/disk_monitor.sh; echo "Exit code: $?"

If your disk usage on / and /tmp is below 80%, you see exit code 0. If either exceeds 80%, you see exit code 1. Either way, the script now tells the truth about what happened.

  • Verify the trap still cleans up by checking for temp files:
ls /tmp/disk_monitor.*

You should see "No such file or directory". The EXIT trap cleaned up after a successful run too.

  • Test Ctrl+C handling by running the script and pressing Ctrl+C immediately:
./scripts/disk_monitor.sh && ls /tmp/disk_monitor.*

Even if you interrupt the script, the EXIT trap fires and removes any temp file that was created. No orphaned files left behind.

✔️ Awesome, I've got everything!

Confirm you have saved scripts/disk_monitor.sh and that shellcheck scripts/disk_monitor.sh produces no output.

ⓧ I'd like to double check the full code

Your scripts/disk_monitor.sh should contain these sections in order. Scroll up and confirm each one is present:

  • Shebang + set -euo pipefail (lines 1-2).
  • Configuration section with THRESHOLD and TMPFILE="".
  • Cleanup section with the cleanup() function and trap cleanup EXIT.
  • Functions section with the updated check_disk() including the df guard, TMPFILE writes, and return codes.
  • Main section with mktemp, the if ! loop over / and /tmp, and the final exit 1 conditional.

Your script is now hardened against all three categories of silent failure. Next up, you will add structured logging and defensive input validation so the script can run safely unattended at 3am.

Add Structured Logging and Defensive Patterns

Your script now stops on errors, catches unset variables, and cleans up temp files automatically. Those are solid defenses for runtime failures.

But what happens when something fails at 3am with no one watching? Without structured logs, you have no way to diagnose what happened after the fact. In this step, you will add timestamped logging with severity levels and defensive input validation that rejects bad arguments before they can cause damage.

In this step, get ready to:
  • Add a structured logging function with timestamps, severity levels, and dual output to terminal and log file.
  • Add defensive input validation that rejects bad arguments immediately.
  • Add signal handling for graceful Ctrl+C interruption and commit the production-ready script.
Add a structured logging function

Production scripts need a consistent way to report what is happening. A log function that tags every message with a timestamp, severity level, and script name makes it possible to trace failures in a log file hours after they happened.

  • In scripts/disk_monitor.sh, find the # --- Configuration --- section at the top of the file (below set -euo pipefail).
  • Replace everything from # --- Configuration --- down to (but not including) # --- Cleanup --- with this code:
# --- Configuration ---
readonly SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_VERSION="1.0.0"
THRESHOLD="${1:-80}"
LOGFILE="${DISK_MONITOR_LOG:-./disk_monitor.log}"
TMPFILE=""

# --- Logging ---
log() {
    local level="$1"
    shift
    local timestamp
    timestamp=$(date '+%Y-%m-%d %H:%M:%S')
    local message="[$timestamp] [$level] [$SCRIPT_NAME] $*"

    case "$level" in
        ERROR|WARN)
            echo "$message" >&2
            ;;
        *)
            echo "$message"
            ;;
    esac

    echo "$message" >> "$LOGFILE"
}

What does this code do?

The configuration section now declares three new variables:

  • SCRIPT_NAME captures the script's filename using basename, so log messages identify which script produced them.
  • SCRIPT_VERSION gives you a version number to track which iteration is running in production.
  • LOGFILE defaults to ./disk_monitor.log but can be overridden by setting the DISK_MONITOR_LOG environment variable.

The log function formats each message as [timestamp] [level] [script_name] message. It routes ERROR and WARN to stderr (so they stand out) and INFO to stdout. Every message also appends to the log file for post-mortem analysis.

Now update the cleanup function to use structured logging instead of silent removal.

  • Find the # --- Cleanup --- section and replace everything from that comment down to and including trap cleanup EXIT with this code:
# --- Cleanup ---
cleanup() {
    local exit_code=$?
    if [[ -n "${TMPFILE:-}" && -f "$TMPFILE" ]]; then
        rm -f "$TMPFILE"
        log "INFO" "Cleaned up temp file: $TMPFILE"
    fi
    if [[ "$exit_code" -ne 0 ]]; then
        log "ERROR" "Script exited with code $exit_code"
    else
        log "INFO" "Script completed successfully"
    fi
    exit "$exit_code"
}

trap cleanup EXIT
trap 'log "WARN" "Received interrupt signal"; exit 130' INT TERM

What changed in cleanup?

The cleanup function now reports what it did. It logs whether it removed a temp file, and whether the script exited successfully or with an error code.

The new INT TERM trap catches Ctrl+C (SIGINT) and termination signals. It logs a WARN message and exits with code 130, the conventional exit code for SIGINT (128 + signal number 2).

  • Save scripts/disk_monitor.sh.
  • Run the script to see structured logging in action:
./scripts/disk_monitor.sh

You should see a mix of the old echo output from check_disk alongside new timestamped messages from cleanup. The final line will read something like [INFO] [disk_monitor.sh] Script completed successfully.

Not seeing log messages?

Make sure the # --- Logging --- section appears BEFORE # --- Cleanup --- in the file. The log function must be defined before cleanup tries to call it.

help me debug why my log function isn't being called from cleanup

Add input validation

Defensive scripts verify their environment before doing real work. If a required command is missing or an argument is garbage, the script should fail loudly and immediately rather than producing corrupt output.

  • In scripts/disk_monitor.sh, find the line trap 'log "WARN" "Received interrupt signal"; exit 130' INT TERM.
  • Add the following validation section directly below that line (before # --- Functions ---):
# --- Validation ---
if ! command -v df > /dev/null 2>&1; then
    log "ERROR" "Required command 'df' not found"
    exit 1
fi

if ! [[ "$THRESHOLD" =~ ^[0-9]+$ ]] || [[ "$THRESHOLD" -lt 1 ]] || [[ "$THRESHOLD" -gt 100 ]]; then
    log "ERROR" "Threshold must be a number between 1 and 100, got: $THRESHOLD"
    exit 1
fi

What does this validation do?

  • command -v df checks that the df utility exists on this system. This is preferred over which on Debian 12, where which writes a deprecation warning to stderr.
  • The regex test [[ "$THRESHOLD" =~ ^[0-9]+$ ]] confirms the threshold is a number. The range check ensures it falls between 1 and 100.
  • Both checks use the log function to produce a structured ERROR message before exiting, so the failure is traceable in the log file.
  • Save scripts/disk_monitor.sh.
  • Test the validation by running the script with a non-numeric threshold:
./scripts/disk_monitor.sh abc

You should see a structured ERROR message like [ERROR] [disk_monitor.sh] Threshold must be a number between 1 and 100, got: abc and the script exits immediately.

  • Confirm the exit code by running:
echo $?

You should see 1, proving the script rejected the bad input.

Script still accepts bad input?

Check that your # --- Validation --- section is placed AFTER the THRESHOLD assignment in Configuration and BEFORE the # --- Functions --- section. If validation runs before THRESHOLD is set, set -u will catch it as unset instead.

help me figure out why my threshold validation isn't catching bad input

Update the script to use logging everywhere

The check_disk function and main section still use raw echo statements. Replacing them with log calls gives every message in the script the same structured format.

  • Find the # --- Functions --- section and replace the entire check_disk function with this updated version:
# --- Functions ---
check_disk() {
    local mount_point="$1"
    local usage

    if ! df "$mount_point" > /dev/null 2>&1; then
        log "ERROR" "Cannot access mount point: $mount_point"
        return 1
    fi

    usage=$(df "$mount_point" | tail -1 | awk '{print $5}' | tr -d '%')

    if [[ "$usage" -gt "$THRESHOLD" ]]; then
        log "WARN" "$mount_point at ${usage}% - EXCEEDS threshold"
        echo "$mount_point: ${usage}% [ALERT]" >> "$TMPFILE"
        return 1
    else
        log "INFO" "$mount_point at ${usage}% - OK"
        echo "$mount_point: ${usage}% [OK]" >> "$TMPFILE"
        return 0
    fi
}

What changed in check_disk?

Each echo that reported status is now a log call with an appropriate severity level. Mount points exceeding the threshold get WARN. Accessible mounts get INFO. Inaccessible mounts get ERROR.

The echo lines that write to $TMPFILE remain unchanged. Those build the summary report file, separate from the log stream.

  • Now find the # --- Main --- section and replace everything from that comment to the end of the file with this code:
# --- Main ---
TMPFILE=$(mktemp /tmp/disk_monitor.XXXXXX)
log "INFO" "Starting disk monitor (threshold: ${THRESHOLD}%, version: $SCRIPT_VERSION)"

alert_count=0
for mount in / /tmp; do
    if ! check_disk "$mount"; then
        ((alert_count++)) || true
    fi
done

log "INFO" "Check complete. Alerts: $alert_count"

if [[ "$alert_count" -gt 0 ]]; then
    log "WARN" "Disk usage alerts detected - review $TMPFILE"
    cat "$TMPFILE"
    exit 1
fi

What changed in main?

  • The old echo header lines are replaced with a single log "INFO" announcing the start, including the threshold and version.
  • After the loop, a log "INFO" summarises how many alerts were found.
  • If alerts exist, cat "$TMPFILE" displays the alert summary before exiting. The cleanup trap will then remove the temp file and log the final exit status.
  • Save scripts/disk_monitor.sh.
  • Before you run the script, predict what the output will look like. Will you see plain text or timestamped messages? How many INFO lines do you expect?
  • Run the final version:
./scripts/disk_monitor.sh

You should see fully structured output with timestamped lines like:

[2026-08-07 14:30:00] [INFO] [disk_monitor.sh] Starting disk monitor (threshold: 80%, version: 1.0.0)
[2026-08-07 14:30:00] [INFO] [disk_monitor.sh] / at 45% - OK
[2026-08-07 14:30:00] [INFO] [disk_monitor.sh] /tmp at 12% - OK
[2026-08-07 14:30:00] [INFO] [disk_monitor.sh] Check complete. Alerts: 0
[2026-08-07 14:30:00] [INFO] [disk_monitor.sh] Script completed successfully
  • Confirm a log file was created by running:
cat disk_monitor.log

You should see the same timestamped messages duplicated in the log file. This is your post-mortem trail for unattended runs.

  • Test the interrupt signal handling by running the script and pressing Ctrl+C quickly:
./scripts/disk_monitor.sh; echo "Exit: $?"

If you press Ctrl+C fast enough, you will see a WARN message about the interrupt signal. The script exits with code 130.

  • Run shellcheck one final time to confirm zero warnings:
shellcheck scripts/disk_monitor.sh

You should see no output, meaning zero warnings. The script is clean.

  • Commit the production-ready script to Git by running:
git add -A && git commit -m "Production-ready disk monitor"

ShellCheck reporting warnings?

The most common issue is quoting. Make sure every variable reference uses double quotes ("$TMPFILE" not $TMPFILE). Also check that local declarations and assignments are on separate lines for timestamp inside the log function.

help me fix shellcheck warnings in my disk monitor script

✔️ Awesome, I've got everything!

Great. Confirm you have saved scripts/disk_monitor.sh and committed it to Git.

ⓧ I'd like to double check the full code

Your scripts/disk_monitor.sh is too long to show in one block. Scroll up and confirm the following sections appear in this exact order:

  • Shebang (#!/usr/bin/env bash) and set -euo pipefail.
  • Configuration section with SCRIPT_NAME, SCRIPT_VERSION, THRESHOLD, LOGFILE, and TMPFILE.
  • Logging section with the log() function using a case statement for stderr routing.
  • Cleanup section with cleanup() using log calls, trap cleanup EXIT, and trap ... INT TERM.
  • Validation section with command -v df and the threshold regex check.
  • Functions section with check_disk() using log instead of echo for status messages.
  • Main section with mktemp, the for loop, log calls for start/complete/alerts, and cat "$TMPFILE" for alert output.

Your disk monitor is now production-ready. It logs every action, rejects bad input, and handles interrupts gracefully. Next up, you will extend it with subcommands and a dry-run mode in the Secret Mission.

Secret mission

Add Subcommands and Dry-Run Mode

Your disk monitor works, but it only does one thing. Real CLI tools have subcommands (like git commit, git push, git log). Can you add check, report, and clean subcommands plus a --dry-run flag that shows what would happen without actually doing it?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • ~/prod-scripts/ directory with Git history (local files).
  • Temp files in /tmp/disk_monitor.* (cleaned up automatically by the trap handler).
  • Log file at ./disk_monitor.log inside your project directory.

Keep everything running

No action needed. Choose this if you plan to continue with Parts 2B-2E of this series.

  • Your ~/prod-scripts/ directory stays on disk with full Git history.
  • The trap handler cleans up temp files automatically each time the script runs.
  • Your log file grows with each run. You can check its size anytime with ls -lh ~/prod-scripts/disk_monitor.log.

Pause - I'll come back to this later

Archive the project to free up active workspace clutter while preserving your work.

  • Create a compressed archive by running:
tar czf ~/Desktop/prod-scripts-backup.tar.gz ~/prod-scripts/
  • Remove any leftover temp files by running:
rm -f /tmp/disk_monitor.*
  • When you are ready to resume, extract the archive with tar xzf ~/Desktop/prod-scripts-backup.tar.gz -C ~/.

Delete - I don't want to use this again

Remove all project files, temp files, and logs permanently.

  • Delete the project directory and all its contents by running:
rm -rf ~/prod-scripts
  • Remove any leftover temp files by running:
rm -f /tmp/disk_monitor.*
  • Confirm everything is gone by running:
ls ~/prod-scripts 2>&1
ls /tmp/disk_monitor.* 2>&1
  • Both commands should return "No such file or directory" errors, confirming the cleanup is complete.

Nice Work!

Nice Work!

You just built a production-grade disk space monitor from scratch. You started with a naive script, watched it fail silently in three different ways, and systematically hardened it into something safe to run unattended at 3am.

Here's what you learned:

  • Diagnosed three categories of silent failure in shell scripts (ignored errors, unset variables, masked pipeline failures) and eliminated all three with set -euo pipefail.
  • Implemented trap-based cleanup that guarantees temp file removal even on crashes or Ctrl+C, using mktemp for safe temp files and trap cleanup EXIT for reliable teardown.
  • Created a reusable structured logging function with timestamps and severity levels, paired with defensive input validation that rejects bad arguments before they can cause damage.
  • Secret Mission: Extended the script with subcommands and a --dry-run mode for safe testing in production environments.

Ready to quiz yourself?