Recover a Broken Linux Service

Diagnose and repair a failing systemd service in WSL 2.

Introduction

30 Second Summary

A computer can look healthy while one quiet task has stopped doing its job. Recovery starts with following the evidence until you find the blocked file or permission.

In this project, you will build OpsWatch as a Linux load monitor on Ubuntu inside Windows Subsystem for Linux version 2. You will deploy it as a restricted systemd service before diagnosing a deliberate failure involving file permissions and applying a durable least-privilege repair.

What You'll Build

You'll watch a working OpsWatch preview become a failed restricted service before your diagnosis brings it back with a continuously updated load report.

By the end of this project, you'll have:

A live OpsWatch preview that displays changing Linux load values plus a rising iteration count in your terminal.
A failed-service incident you can trace through unit state, journal logs, account identity, ownership, and permissions.
A recovered OpsWatch service that continuously updates /var/lib/opswatch/status.txt under a dedicated account. You can control its startup and collect fresh health evidence from the terminal.
Secret Mission: Add an automatic restart policy and prove that OpsWatch recovers from a simulated permissions fault.

Are there any prerequisites?

You need a Windows computer with either WSL or Ubuntu already installed. Step 1 checks every version and access requirement before you build.

Before We Start

OpsWatch reads Linux kernel load data from /proc/loadavg. Before the hands-on work begins, this checkpoint locks in why it runs as a dedicated non-login system user to follow least privilege.

Verify the Linux Administration Lab

OpsWatch needs a predictable Linux host before you manage it as a service. The required lab uses Ubuntu 24.04 LTS on Windows Subsystem for Linux (WSL) 2.

Systemd must run as the system manager. Your Linux account also needs sudo access before it can administer services.

In this step, get ready to:

Inspect WSL from Windows PowerShell.
Prepare the required Ubuntu distribution on the correct WSL generation.
Verify Bash, GNU coreutils, systemd, and your administrative access.

Inspect the WSL environment

Windows PowerShell reports the installed WSL version. It also lists the WSL generation assigned to each Linux distribution.

Press the Windows key to open the Windows search bar.
Type Windows PowerShell into the search bar.
Press Enter to open Windows PowerShell.
Inspect WSL and its registered distributions by running these commands:

wsl --version
wsl.exe --list --verbose

What do these commands check?

The first command prints details about the WSL installation on Windows.
The second command lists each installed distribution with its assigned WSL generation.
Your target distribution is Ubuntu-24.04 on WSL 2.

PowerShell cannot inspect WSL?

Confirm that you opened Windows PowerShell on the Windows side of your computer. A shell inside Ubuntu cannot run these Windows setup checks in the same way.

Help me troubleshoot my WSL inspection commands. You can also share the non-sensitive command output with the NextWork community.

Prepare the required Ubuntu distribution

Your PowerShell results determine which setup path you need. Use the outcome below that matches your distribution.

✔️ The lab matches the requirement

Your output lists Ubuntu-24.04 on WSL 2. That distribution is ready for the Ubuntu checks.

Keep Windows PowerShell available for the screenshot checkpoint below.

ⓧ Ubuntu uses the older WSL generation

Your Ubuntu distribution exists on the older WSL generation. The lab requires WSL 2.

Convert Ubuntu-24.04 to WSL 2 by running this command:

wsl.exe --set-version Ubuntu-24.04 2

What does this command do?

This command changes the installed Ubuntu-24.04 distribution to WSL 2. PowerShell reports progress while Windows completes the conversion.

Predict which WSL generation PowerShell will report after the conversion.
Confirm the distribution generation by running this command:

wsl.exe --list --verbose

What should you see?

You should see Ubuntu-24.04 listed with WSL 2. This confirms that the distribution now uses the required generation.

Conversion did not complete?

Let the conversion finish before running the inspection command again. If it stops with an environment-related failure, update WSL through the missing-distribution path.

Help me troubleshoot the WSL conversion. You can also share the non-sensitive command output with the NextWork community.

ⓧ The required Ubuntu distribution is missing

The lab needs a usable Ubuntu 24.04 LTS distribution. Updating WSL first prepares the Windows environment for that installation.

Update WSL by running this command:

wsl --update

What does this command do?

This command updates the WSL installation on Windows. The update prepares WSL to run the required Ubuntu distribution with current systemd support.

The installer manages the required Windows features and the distribution download. A restart request is expected when Windows needs to finish enabling those features.

Install the required Ubuntu distribution by running this command:

wsl --install Ubuntu-24.04

What does this command do?

This command installs the distribution registered as Ubuntu-24.04. Its first launch asks you to create the default Linux account.

Restart Windows if PowerShell requests it.
Press the Windows key to open the Windows search bar.
Type Ubuntu 24.04 LTS into the search bar.
Press Enter to launch Ubuntu.

The first launch creates your default Linux account. This account receives permission to use sudo for administrative tasks.

Choose a Linux username when Ubuntu requests one.
Choose a Linux password when Ubuntu requests one.
Confirm the Linux password when Ubuntu requests it again.
Return to Windows PowerShell after the Ubuntu shell opens.
Predict which WSL generation PowerShell will show for the new distribution.
Confirm the installation by running this command:

wsl.exe --list --verbose

What should you see?

You should see Ubuntu-24.04 listed with WSL 2. This proves that the required distribution is installed on the correct generation.

Ubuntu still missing from the list?

Check whether PowerShell requested a Windows restart during installation. Complete that restart before launching Ubuntu again.

Help me troubleshoot the Ubuntu installation. You can also share the non-sensitive command output with the NextWork community.

Verify sudo, tools, and systemd

The remaining checks happen inside Ubuntu. They confirm that your account can perform administrative work and that every command required by the lab is available.

Switch to the Ubuntu 24.04 LTS window if it is already running.
Press the Windows key if the Ubuntu window is closed.
Type Ubuntu 24.04 LTS into the Windows search bar.
Press Enter to launch Ubuntu.

Your Linux password protects administrative actions inside Ubuntu. A password prompt is expected during the first sudo check.

Validate your administrative access by running this command:
Enter your Linux password if Ubuntu requests it.

sudo -v

What does this command check?

The validation option asks sudo to authenticate your account without running another command. A return to the shell without an error confirms that your account has administrative access.

That quiet return is a success signal. Your Linux account can now perform the administrative work used throughout the lab.

Sudo access was not accepted?

Retry with the Linux password created for this Ubuntu distribution. Windows account passwords do not authenticate the Linux account.

Help me troubleshoot my sudo access. You can also share the non-sensitive result with the NextWork community.

The final checks confirm the exact command-line environment used by OpsWatch. The system status check also proves that systemd is running as Ubuntu's system manager.

Predict whether all four commands will confirm the required lab environment.
Verify Bash, systemd, GNU install, and the system manager by running these commands:

/usr/bin/bash --version
systemctl --version
/usr/bin/install --version
systemctl status

What do these checks prove?

Bash should report version 5.2.21.
Systemd should report version 255.
GNU install should report GNU coreutils version 9.4.
The final command should display systemd's system status.
That status confirms systemd is running as the system manager.

Press Q to return to the shell if the system status remains open in a pager.

A tool or system status is missing?

Confirm that the terminal belongs to Ubuntu 24.04 LTS. A different distribution can provide different tool versions or systemd behavior.

Help me troubleshoot the Ubuntu lab checks. You can also share the non-sensitive command output with the NextWork community.

Your Ubuntu lab now has the required distribution, WSL generation, administrative access, command-line tools, and running system manager. Next up, you will build the OpsWatch monitor and produce its first live report.

Build the OpsWatch Monitor

Your Ubuntu lab is ready for service administration. The next task is proving the monitor can work on its own.

A manual run gives you a visible working baseline before service accounts introduce new failure modes. You'll build OpsWatch in Bash before watching it update a report in your home directory.

In this step, get ready to:

Create the OpsWatch script on the native Linux filesystem.
Run the monitor until it writes at least two report updates.
Inspect the preview report to connect its fields to kernel load data.

Create the OpsWatch script

Files under /usr/local/bin stay on the native Linux filesystem. This keeps ownership behavior predictable for later service work.

The target directory is system-managed, so sudo may request your Linux password. Hidden password input is expected.

Return to the Ubuntu terminal from earlier.
Start writing /usr/local/bin/opswatch.sh as an administrator by running this command:

sudo tee /usr/local/bin/opswatch.sh

What is tee doing?

The tee command copies your next input into /usr/local/bin/opswatch.sh. It also echoes that input in the terminal so you can see what was written.

Paste the script below into the waiting terminal:

set -eu

output_file=${1:-/var/lib/opswatch/status.txt}
iteration=0

while true; do
  iteration=$((iteration + 1))
  IFS=' ' read -r load_1m load_5m load_15m process_counts last_pid < /proc/loadavg

  {
    printf 'service=opswatch\n'
    printf 'iteration=%s\n' "$iteration"
    printf 'load_1m=%s\n' "$load_1m"
    printf 'load_5m=%s\n' "$load_5m"
    printf 'load_15m=%s\n' "$load_15m"
    printf 'processes=%s\n' "$process_counts"
    printf 'last_pid=%s\n' "$last_pid"
  } > "$output_file"

  printf 'Updated %s (iteration %s)\n' "$output_file" "$iteration"
  /usr/bin/sleep 10
done

What does this script do?

The set -eu line makes Bash stop when a command fails or an unset variable is expanded.
The output_file variable uses the first argument as the report path.
The iteration variable counts how many reports the current process has written.
The read -r command separates the five fields from /proc/loadavg into named variables.
The grouped printf calls replace the report with current values during each loop.
The final printf makes every successful update visible in the terminal.
The /usr/bin/sleep 10 line creates a ten-second interval between updates.

Press Ctrl+D on an empty line to finish writing the script.
Confirm that Ubuntu returns you to the shell prompt.

You should see the pasted script echoed before the shell prompt returns. That output confirms that tee received the complete file content.

Still waiting for the shell prompt?

Press Ctrl+D again while the cursor is on an empty line.
Rerun the tee command with sudo if Ubuntu reports a permissions problem.

Ask for help with writing the OpsWatch script with tee.

✔️ Awesome, I've got everything!

Your tee command has finished. The OpsWatch script is ready for its manual run.

ⓧ I'd like to double check the full code

Compare /usr/local/bin/opswatch.sh with this full reference:

set -eu

output_file=${1:-/var/lib/opswatch/status.txt}
iteration=0

while true; do
  iteration=$((iteration + 1))
  IFS=' ' read -r load_1m load_5m load_15m process_counts last_pid < /proc/loadavg

  {
    printf 'service=opswatch\n'
    printf 'iteration=%s\n' "$iteration"
    printf 'load_1m=%s\n' "$load_1m"
    printf 'load_5m=%s\n' "$load_5m"
    printf 'load_15m=%s\n' "$load_15m"
    printf 'processes=%s\n' "$process_counts"
    printf 'last_pid=%s\n' "$last_pid"
  } > "$output_file"

  printf 'Updated %s (iteration %s)\n' "$output_file" "$iteration"
  /usr/bin/sleep 10
done

What should match?

The first section defines the output path plus the starting iteration.
The loop reads kernel load data before replacing the report.
The final lines print a visible update before pausing for ten seconds.

Run the monitor manually

Running the script directly isolates its own behavior from service users plus protected directories. This gives you a result you can compare with the managed service later.

The second update arrives about ten seconds after the first. A quiet pause during that interval is expected.

Before you run it, how many update messages do you expect to see if the script keeps looping?

Run OpsWatch with a preview file in your home directory by entering this command:

/usr/bin/bash /usr/local/bin/opswatch.sh "$HOME/opswatch-preview.txt"

What does this command prove?

The command asks Bash to run /usr/local/bin/opswatch.sh. The preview path becomes the script's first argument.

Writing inside your home directory tests the monitor with a location your current Linux account can access. This creates the working baseline for the later permissions investigation.

Wait until the terminal displays the second Updated message.
Press Ctrl+C to stop the foreground monitor.

That's your first OpsWatch success: the script repeatedly wrote its report without systemd managing the process.

No update messages?

Compare /usr/local/bin/opswatch.sh with the full-code tab if Bash exits before the first update.
Check that the preview path begins with $HOME if the script reports a write problem.

Ask for help with a stopped or silent OpsWatch manual run.

Inspect the preview report

The terminal messages prove the loop ran. The report shows exactly what each successful iteration captured from the kernel.

Before you print the file, which line should prove that the loop completed more than once?

Print the completed preview report by running this command:

/usr/bin/cat "$HOME/opswatch-preview.txt"

How does the report map to loadavg?

The service field identifies OpsWatch as the report source.
The iteration field records the latest completed loop.
The load_1m field stores the first value from /proc/loadavg.
The load_5m field stores the second value from /proc/loadavg.
The load_15m field stores the third value from /proc/loadavg.
The processes field stores the runnable plus total process counts.
The last_pid field stores the most recently created process ID.

You should see service=opswatch near the top. The iteration value should be 2 or higher.

Every remaining report line should contain a current kernel value. Together, these lines prove that OpsWatch produced a complete manual report.

Preview file missing or incomplete?

Rerun the monitor from the previous substep if the preview file does not exist.
Wait for the second update before pressing Ctrl+C if the iteration value is below 2.
Compare the report-writing section with the full-code tab if any field is missing.

Ask for help with an incomplete OpsWatch preview report.

Your monitor now has a proven manual baseline. Next, you'll deploy it under a dedicated account to test the service permissions boundary.

Deploy a Failing Service

OpsWatch now reads live kernel load data from your Ubuntu account. It also writes a changing preview report.

Running the same script through a restricted account exposes a real file-permissions boundary. This step turns that boundary into a controlled incident with evidence from systemd service state and journal logs.

In this step, get ready to:

Create a dedicated non-login account for OpsWatch.
Prepare the root-owned state directory and service unit.
Start the unit and capture evidence from its controlled failure.

Create the dedicated service account

A dedicated system user gives OpsWatch a narrow identity. The account has no interactive login.

Create the OpsWatch system account and matching group by running this command:

sudo adduser --system --group opswatch

What Did Adduser Create?

The --system option creates the non-login opswatch account.
The --group option creates a matching primary group.
The account has no password. Its login shell is /usr/sbin/nologin.

Inspect the new account identity by running this command:

id opswatch

What Does ID Confirm?

The id command prints the account's user information. It also shows that the primary group is named opswatch.

That account is ready. OpsWatch now has its own restricted identity for service execution.

Account Creation Rejected?

If the command reports that the account already exists, use the id output to confirm that it has a matching group. If sudo refuses the command, confirm that you are using the sudo-capable Ubuntu account from the lab setup.

Help me check the OpsWatch system account.

Prepare the state directory and unit

OpsWatch's default report path sits inside /var/lib/opswatch. This directory becomes the boundary between the root administrator and the restricted service identity.

Create the root-owned state directory with mode 0755 by running this command:

sudo install -d -o root -g root -m 0755 /var/lib/opswatch

What Did Install Set?

The -d option creates /var/lib/opswatch as a directory.
The -o root option makes root the owner.
The -g root option assigns the root group.
The -m 0755 option gives the owner write access. Group members and other users receive read and execute access.

Directory Command Rejected?

If sudo rejects the command, confirm that your Ubuntu account has administrator access. Rerunning the command applies the requested owner, group, and permission mode to the directory.

Help me create the OpsWatch state directory.

A systemd service unit defines the identity used to launch a process. It also defines the executable that systemd starts.

This is the controlled incident version of the unit. Keep it exactly as shown so the failure produces consistent evidence.

This terminal input flow can feel unfamiliar. sudo may request your Linux password before tee waits for the unit text.

Start writing /etc/systemd/system/opswatch.service by running this command:

sudo tee /etc/systemd/system/opswatch.service

How Does This Input Work?

The tee command now reads text from your terminal. It copies the text you paste into /etc/systemd/system/opswatch.service.

Enter your Linux password if sudo requests it.
Paste this incident service unit into the waiting terminal:

[Unit]
Description=OpsWatch load monitor

[Service]
Type=exec
User=opswatch
Group=opswatch
ExecStart=/usr/bin/bash /usr/local/bin/opswatch.sh

[Install]
WantedBy=multi-user.target

What Does This Unit Define?

The [Unit] section gives the service its OpsWatch description.
Type=exec lets systemd track process setup failures for this long-running service.
User=opswatch sets the process user. Group=opswatch sets its process group.
ExecStart= launches the existing Bash monitor without a custom output argument.
WantedBy=multi-user.target makes the unit eligible for enablement later.

Finish writing the unit by pressing Ctrl+D.
Print the saved unit for comparison by running this command:

/usr/bin/cat /etc/systemd/system/opswatch.service

What Should You See?

The terminal should print the complete incident unit. Its contents should match the file below exactly.

Unit Content Does Not Match?

Run the tee command again if the file is empty.
Compare every printed line with the full unit in the double-check tab.

Help me correct the OpsWatch service unit.

✔️ Awesome, I've got everything!

Your saved unit matches the incident version. The service is ready for its controlled start.

ⓧ I'd like to double check the full code

Compare the printed unit with this exact file:

[Unit]
Description=OpsWatch load monitor

[Service]
Type=exec
User=opswatch
Group=opswatch
ExecStart=/usr/bin/bash /usr/local/bin/opswatch.sh

[Install]
WantedBy=multi-user.target

This is the complete service unit for this step. It contains the service identity and the existing monitor command.

Start the incident and collect evidence

Systemd keeps a loaded model of its unit files. A daemon reload makes the new OpsWatch definition available to the service manager.

Reload the systemd unit files by running this command:

sudo systemctl daemon-reload

What Does Daemon Reload Do?

The command reloads unit files from disk. It also rebuilds systemd's dependency tree.

The next start is the designed incident. Its first version needs to fail so you can capture real troubleshooting evidence.

Start opswatch.service by running this command:

sudo systemctl start opswatch.service

What Happens During Start?

Systemd launches /usr/bin/bash as the opswatch service user. Service output and errors are captured in the journal.

Before you inspect the result, predict whether a script that worked manually will still run under the new service identity.

Wait one second and inspect the failure state by running these commands:

/usr/bin/sleep 1
systemctl is-failed opswatch.service

The Failure Is Intentional

You should see failed. This proves that systemd launched the unit but the process could not keep running.

Read the latest unit-specific journal evidence by running this command:

sudo journalctl -u opswatch.service -n 20 --no-pager

What Does the Journal Reveal?

You should see an entry indicating that /var/lib/opswatch/status.txt could not be written because of its permissions. The script worked under your account but failed under the restricted opswatch identity.

The directory belongs to root with mode 0755. Only its owner has write access.

Service Did Not Reach Failed State?

Compare the installed unit with the incident version in the double-check tab.
Confirm that /var/lib/opswatch was created by the directory command from this step.
Repeat the daemon reload before starting the service again.

Help me reproduce the OpsWatch service failure.

That controlled failure is working exactly as designed. You now have a reproducible incident with service-state evidence and a unit-specific journal trail.

You now have a failed service backed by evidence from systemctl and the journal. Next, you'll trace that evidence to the permissions boundary and repair the service.

Diagnose and Repair the Service

OpsWatch worked during its manual test. The same script failed when the service ran under its restricted account.

Which identity owns the process? Which file permissions control the report directory?

The durable repair uses systemd to manage persistent state for the configured service user. This preserves the least-privilege boundary.

In this step, get ready to:

Identify the permission mismatch from the existing evidence.
Configure systemd to manage OpsWatch's persistent state.
Verify the repaired service with runtime evidence.

Trace the permission boundary

A permission failure becomes easier to solve when each claim has evidence. Collect one snapshot that connects identity to ownership.

Collect the permission evidence by running these commands:

id opswatch
ls -ld /var/lib/opswatch
/usr/bin/cat /etc/systemd/system/opswatch.service
sudo journalctl -u opswatch.service -n 20 --no-pager

What does the evidence prove?

The id output confirms that opswatch is the service account. It also shows the matching group.
The ls output shows root as the owner. It also shows root as the group. The drwxr-xr-x mode gives write access only to the owner.
The unit file confirms that the service process uses opswatch through User= and Group=.
The journal ties that ownership mismatch to the failed report write.

The opswatch process cannot create status.txt because only root can write to the 0755 directory.

Evidence looks different?

Confirm that the directory path is /var/lib/opswatch. The expected pre-repair state has owner root, group root, and mode 0755.

Help me connect the OpsWatch identity, directory permissions, unit settings, and journal evidence.

Let systemd manage persistent state

The StateDirectory=opswatch setting tells systemd to create /var/lib/opswatch for the configured service identity. Existing mismatched ownership is corrected when the unit starts.

The StateDirectoryMode=0750 setting limits access to the service owner and group. Other users receive no access through this mode.

Use the file-editing workflow from the previous step to update /etc/systemd/system/opswatch.service.
Replace the current unit content with this configuration:

[Unit]
Description=OpsWatch load monitor

[Service]
Type=exec
User=opswatch
Group=opswatch
StateDirectory=opswatch
StateDirectoryMode=0750
ExecStart=/usr/bin/bash /usr/local/bin/opswatch.sh

[Install]
WantedBy=multi-user.target

What does this unit change?

The User= and Group= settings keep the process under the restricted opswatch identity.
The StateDirectory=opswatch setting gives systemd responsibility for the persistent state directory.
The StateDirectoryMode=0750 setting gives full access to the owner. It gives read and execute access to the group. It gives no access to other users.
The ExecStart= setting continues to run the existing OpsWatch script.

Save /etc/systemd/system/opswatch.service.
Check the [Service] section in your saved file.

You should see the two state-directory settings between Group=opswatch and ExecStart=/usr/bin/bash /usr/local/bin/opswatch.sh.

Unit settings not matching?

Compare the capitalization of StateDirectory= and StateDirectoryMode=. Confirm that both settings remain inside the [Service] section.

Help me check the repaired OpsWatch unit file.

✔️ Awesome, I've got everything!

Your saved unit now gives systemd responsibility for the OpsWatch state directory.

ⓧ I'd like to double check the full code

Compare your saved /etc/systemd/system/opswatch.service with this complete unit:

[Unit]
Description=OpsWatch load monitor

[Service]
Type=exec
User=opswatch
Group=opswatch
StateDirectory=opswatch
StateDirectoryMode=0750
ExecStart=/usr/bin/bash /usr/local/bin/opswatch.sh

[Install]
WantedBy=multi-user.target

Reload and verify the repair

The service manager still holds the old unit definition in memory. A reload followed by a restart applies the new directory policy.

Apply the repaired unit configuration by running these commands:

sudo systemctl daemon-reload
sudo systemctl restart opswatch.service
sudo journalctl -u opswatch.service -n 20 --no-pager

What happens during the repair?

The daemon-reload command makes systemd reread the edited unit.
The restart applies the state-directory policy. Systemd changes the managed directory to the configured service identity.
The journal should include a fresh Updated entry from OpsWatch. This proves the script can write its report.

Restart still failing?

Check the journal for the setting or path mentioned near the latest failure. Compare the saved unit with the full-code tab before repeating the reload.

Help me diagnose why the repaired OpsWatch service still fails.

Before you verify the repair, what ownership, service state, and report fields do you expect to see?

Verify the repaired directory and service by running these commands:

ls -ld /var/lib/opswatch
systemctl is-active opswatch.service
/usr/bin/cat /var/lib/opswatch/status.txt

What should you see?

The directory listing should show owner opswatch and group opswatch. Its mode should appear as drwxr-x---, which represents 0750.
The service state command should print active.
The report should contain service=opswatch, iteration, load_1m, load_5m, load_15m, processes, and last_pid fields.

Missing the active report?

Confirm that the unit file contains both state-directory settings. Check that ExecStart= still points to /usr/local/bin/opswatch.sh.

Help me troubleshoot the final OpsWatch verification.

You recovered OpsWatch with its least-privilege boundary intact. Next, you'll practice controlling its lifecycle and checking its health like an administrator.

Operate OpsWatch as an Administrator

Your permissions repair has OpsWatch writing reports again. Routine administration now depends on proving that you can control its lifecycle.

In this step, you'll use systemd lifecycle checks to control OpsWatch. You'll use the journal to collect evidence.

In this step, get ready to:

Enable OpsWatch for startup through its configured target.
Stop and start the service to test its lifecycle.
Confirm service health through state checks, journal entries, and the current report.

Enable OpsWatch for startup

A service's enablement state controls whether systemd hooks it into a startup target. Its active state describes whether its process is running now.

Enable opswatch.service for its configured startup target by running this command:

sudo systemctl enable opswatch.service

What Does Enable Change?

The systemctl enable command reads the existing WantedBy=multi-user.target setting. It creates the startup link that registers OpsWatch with that target.

You should see output confirming that a link was created for opswatch.service.

Before you run the checks, consider whether enablement changes the service's current running state.

Compare startup registration with the current runtime state by running these commands:

systemctl is-enabled opswatch.service
systemctl is-active opswatch.service

How Do These States Differ?

The is-enabled check reports whether the unit is registered to start through its configured target.
The is-active check reports whether the service process is running now.

You'll see enabled followed by active. This confirms that opswatch.service is enabled for multi-user.target.

Enablement Check Unexpected?

Confirm that /etc/systemd/system/opswatch.service still contains its Install section.
Confirm that WantedBy=multi-user.target remains in that section.

Help me diagnose why OpsWatch is not reporting enabled and active.

Test the stop and start lifecycle

A controlled stop and start proves that you can change the runtime state without losing startup registration. It also confirms that OpsWatch resumes writing after a normal start.

Before you stop OpsWatch, consider which state should change while its startup registration remains intact.

Test the stopped state by running these commands:

sudo systemctl stop opswatch.service
systemctl is-enabled opswatch.service
systemctl is-active opswatch.service

What Does the Stop Test Prove?

The stop command deactivates the current OpsWatch process.
The enablement check confirms that its startup registration remains in place.
The activity check confirms that no OpsWatch process is currently running.

You'll see enabled followed by inactive. The unit remains configured for startup while its process is stopped.

Service Still Active?

Wait a moment before rerunning the state checks.
Confirm that each command uses the unit name opswatch.service.

Help me troubleshoot an OpsWatch service that remains active after a stop command.

Before you start OpsWatch again, consider what the report should do when the process returns.

Test the started state and its report output by running these commands:

sudo systemctl start opswatch.service
systemctl is-active opswatch.service
/usr/bin/cat /var/lib/opswatch/status.txt

What Does the Start Test Prove?

The start command launches OpsWatch through its existing service unit.
The activity check confirms that the service process is running.
The report confirms that the process can still write to its systemd-managed state directory.

You'll see active followed by the current report. The report includes service=opswatch plus the latest iteration and load fields.

Report Missing After the Start?

Confirm that the activity check reports active.
Confirm that the report path is /var/lib/opswatch/status.txt.
Inspect the unit journal for recent startup or write failures.

Help me diagnose why OpsWatch is active but its status report is missing or stale.

Run the final health check

A final operator health check combines independent signals. Each signal answers a different operational question.

Before you run the final check, consider whether all four signals should agree after your stop and start test.

Collect the final service evidence by running these commands:

systemctl is-enabled opswatch.service
systemctl is-active opswatch.service
sudo journalctl -u opswatch.service -n 10 --no-pager
/usr/bin/cat /var/lib/opswatch/status.txt

How Does This Health Check Work?

The enablement check proves that OpsWatch is registered for startup.
The activity check proves that the service process is currently running.
The unit-specific journal shows recent messages from OpsWatch.
The report shows the current load data written by the application.

You'll see enabled followed by active. You'll also see recent Updated entries plus the current OpsWatch report.

What Does Enabled Mean in WSL?

The enabled unit starts when the Ubuntu Windows Subsystem for Linux (WSL) instance starts. A systemd service does not keep the WSL instance alive after WSL itself shuts down.

That's the administration loop complete. OpsWatch is enabled for startup and active after a successful stop and start test.

Secret mission

Make OpsWatch Self-Healing

Add an automatic restart policy to OpsWatch. Then break its state directory ownership and prove systemd restores the service without a manual restart.

Clean Up Your Resources

Your Ubuntu distribution on Windows Subsystem for Linux holds every resource from this lab at no paid cost. Your next choice is to keep the service running, pause it for later, or delete its local resources.

Resources you used:

The enabled opswatch.service service.
The systemd unit at /etc/systemd/system/opswatch.service.
The monitor script at /usr/local/bin/opswatch.sh.
The manual preview report at $HOME/opswatch-preview.txt.
The persistent state directory at /var/lib/opswatch.
The live report at /var/lib/opswatch/status.txt.
The non-login opswatch system user.
The matching opswatch system group.
The enablement link created for opswatch.service.

Keep everything running

Keeping everything requires no cleanup. Choose this option if you want to keep testing service recovery or watching Linux load data.

OpsWatch remains enabled in Ubuntu.
Its process stays active while the WSL instance runs.
The five-second automatic restart policy remains available for future failures.
The service does not keep the WSL instance alive after WSL shuts down.
The script, unit, reports, system user, and system group remain available.

Pause - I'll come back to this later

Pausing stops the active monitor process. Your script, unit, reports, account, enablement setting, and restart policy remain available.

Stop opswatch.service by running:

sudo systemctl stop opswatch.service

What does this command do?

The stop action ends the active OpsWatch process.

The service remains enabled for the next Ubuntu startup.

Confirm that OpsWatch is paused by running:

systemctl is-active opswatch.service

What should I see?

The inactive result confirms that the service process has stopped.

Return to the service start workflow from Step 5 when you want OpsWatch running again.

Delete - I don't want to use this again

This option permanently removes the OpsWatch service, files, state, account, and group. Your Ubuntu distribution and other Linux files stay untouched.

Disable the service and stop its process by running:

sudo systemctl disable --now opswatch.service

What does this command remove?

The command removes the enablement link for opswatch.service.

The --now option also stops the active process before cleanup continues.

Remove the unit, script, preview report, and persistent state by running:

sudo rm /etc/systemd/system/opswatch.service /usr/local/bin/opswatch.sh "$HOME/opswatch-preview.txt"
sudo rm -r /var/lib/opswatch

What do these commands remove?

The first command removes the custom service unit, monitor script, and manual preview report.
The second command removes the persistent state directory and its live status report.

Remove the dedicated system user and matching system group by running:

sudo deluser --system opswatch
sudo delgroup --system opswatch

What happens to the service identity?

The first command removes the non-login opswatch system account.

The second command removes the matching system group.

Refresh the systemd unit registry by running:

sudo systemctl daemon-reload

Why reload systemd?

The reload makes systemd rebuild its unit information after the custom service file has been removed. OpsWatch is now fully cleared from the lab.

Nice Work!

You did it! OpsWatch now runs as a least-privilege Linux service on WSL 2 with automatic recovery from the permissions fault you simulated.

You've learned how to:

Build a Bash load monitor that turns live kernel data from /proc/loadavg into a changing status report.
Deploy OpsWatch as a least-privilege systemd service under the dedicated opswatch account.
Diagnose the permissions failure from the service identity, directory ownership, permission mode, and journal evidence. Repair persistent state access with StateDirectory= plus StateDirectoryMode=. Verify the recovered service through lifecycle checks.
Secret Mission: Configure an automatic restart policy with Restart=on-failure plus RestartSec=5s. Prove OpsWatch recovers from a simulated ownership fault without a manual restart.

Ready to quiz yourself?