Build a Concurrent URL Health Checker

Build a CLI worker pool in Go using goroutines, channels, and context.

Introduction

30 Second Summary

Ever clicked a link and waited forever, unsure whether the site is down or just slow? Now imagine being responsible for monitoring dozens of URLs at once. Checking them one by one would take forever.

In this project, you will build a command-line tool in Go that checks whether multiple websites are up or down, all at the same time using concurrent workers. You will learn the exact concurrency patterns that power job queues, batch processors, and background workers in production backend systems.

What You'll Build

A concurrent URL health checker that distributes HTTP requests across a pool of worker goroutines and reports the status, response time, and error category for each URL.

By the end of this project, you'll have:

  • A working Go program that checks multiple URLs simultaneously using a worker pool of goroutines and channels.
  • Timeout and error handling using Go's context system that classifies failures into clear categories like timeouts, DNS errors, and connection refused.
  • A measurable concurrency speedup you can observe by running the tool with different worker counts.
  • Secret Mission: Add graceful shutdown with OS signal handling so pressing Ctrl+C cleanly drains in-flight workers before exiting.

Are there any prerequisites?

You need familiarity with at least one programming language (a TypeScript or JavaScript background is assumed). No prior Go experience is required.

Before We Start

Before We Start...

Install Go and Set Up Your Project

Go is a compiled language designed for building fast, concurrent backend systems. Before writing any Go code, you need the Go toolchain installed on your machine and a project directory initialized.

In this step, you'll install Go, set up VS Code with the Go extension, and create your first Go project to confirm everything works together.

In this step, get ready to:
  • Install Go 1.26 and verify it runs.
  • Set up VS Code with the Go extension.
  • Create and run your first Go program.
Install Go 1.26 on macOS

Go compiles your code into a standalone binary that runs without a runtime or interpreter. To use it, you need the Go toolchain installed on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Check if Go is already installed by running this command:
go version

Check the output in your terminal, then follow the tab that matches what you see.

✔️ I see go1.26 or higher

Your output looks something like:

go version go1.26.4 darwin/arm64

Go is ready. Skip ahead to installing the VS Code Go extension below.

ⓧ I see an older version

If you see a version below 1.26 (for example go1.22.5), you need to upgrade. The new installer will replace your existing version.

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.4.darwin-arm64.pkg (Apple Silicon) or go1.26.4.darwin-amd64.pkg (Intel Mac).
  • Double-click the downloaded .pkg file and follow the installer prompts.
  • Close your current Terminal window and open a new one. This ensures the updated PATH takes effect.
  • Verify the upgrade by running:
go version

You should now see go version go1.26.4 darwin/arm64 (or darwin/amd64 for Intel).

ⓧ Command not found

Go is not installed yet. Let's fix that.

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.4.darwin-arm64.pkg (Apple Silicon) or go1.26.4.darwin-amd64.pkg (Intel Mac).
  • Double-click the downloaded .pkg file and follow the installer prompts. The installer places Go at /usr/local/go and adds it to your PATH.
  • Close your current Terminal window and open a new one. This picks up the PATH changes the installer made.
  • Verify the installation by running:
go version

You should see go version go1.26.4 darwin/arm64 (or darwin/amd64 for Intel).

Still seeing command not found?

Make sure you opened a brand new Terminal window after the installer finished. If it still doesn't work, the PATH may not have been set automatically.

Add Go to your PATH manually by running:

echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.zshrc
source ~/.zshrc

Then try go version again. Help me fix my Go PATH issue

What is Go?

Go is an open-source language created at Google for building fast, reliable backend systems. It compiles to a single binary with no external dependencies, making deployment straightforward.

Go's standout feature is built-in concurrency through goroutines and channels. You'll use both heavily in this project.

Now let's set up VS Code with Go language support.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • Open the Extensions view by pressing Cmd+Shift+X (macOS) or Ctrl+Shift+X (Windows).
  • Type Go in the search box.
  • Find the extension published by Go Team at Google (extension ID: golang.go) and click Install.

Why the Go extension?

The Go extension provides autocompletion, inline error highlighting, format-on-save, and auto-imports. It makes writing Go feel as smooth as a dynamically typed language while keeping all the safety of a compiled one.

Create your project directory and initialize a Go module

Every Go project lives inside a module. A module is a directory with a go.mod file that tracks the project's name and Go version.

  • Switch back to your Terminal.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir url-checker
cd url-checker
  • Confirm the folder exists by running:
ls

You should see the url-checker folder listed in the output (or an empty listing if you're already inside it).

  • Initialize a Go module by running:
go mod init url-checker

What does this command do?

go mod init url-checker creates a go.mod file that declares this directory as a Go module named url-checker. Go uses this file to track your project's dependencies and the Go version it targets.

Now let's write your first Go program to confirm the toolchain works end to end.

  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the url-checker folder, and click Open.
  • In the Explorer sidebar on the left, click the new file icon (a page with a + sign).
  • Name the file main.go and press Enter.
  • Paste the following code into main.go:
package main

import "fmt"

func main() {
	fmt.Println("Hello, World!")
}

What does this code do?

package main tells Go this is an executable program (not a library). Every Go executable must have a main package with a main() function.

import "fmt" brings in Go's formatting package, which provides Println for printing to the terminal. Go includes a rich standard library so you rarely need third-party packages.

  • Save main.go by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Open the integrated terminal in VS Code by pressing Ctrl+` (backtick key, same on macOS and Windows).
  • Run your program by running this command:
go run main.go

You should see Hello, World! printed in the terminal.

Don't see Hello, World! in the output?

Make sure you saved main.go before running the command. Unsaved changes won't be compiled.

Check that your terminal is in the url-checker folder. Run pwd to confirm the path ends with /url-checker.

Help me fix my Go program

✔️ Awesome, I've got everything!

Great. Double check you've saved your file, then move on to the next substep.

ⓧ I'd like to double check the full code

Here is the full main.go file:

package main

import "fmt"

func main() {
	fmt.Println("Hello, World!")
}

And your go.mod file (generated by go mod init) should look like this:

module url-checker

go 1.26
Familiarize yourself with the Go development loop

Go has two ways to run your code. go run compiles and executes in one step (great for development). go build compiles a standalone binary you can distribute or deploy.

  • Notice how VS Code shows type information when you hover over fmt.Println. The Go extension provides this automatically.
  • Compile your program into a binary by running this command in the integrated terminal:
go build

What does go build do?

go build compiles your code into a standalone binary named after your module (url-checker). This binary has no dependencies. You can copy it to any machine with the same OS and architecture and it will run.

  • Run the compiled binary directly:
./url-checker

You should see Hello, World! printed again. This time it ran from the compiled binary, not from go run.

Permission denied when running ./url-checker?

Make sure go build completed without errors first. Check that a file named url-checker exists by running ls in the terminal.

If the file exists but won't run, make it executable:

chmod +x url-checker

Help me run my Go binary

  • Try changing "Hello, World!" to "URL Checker starting..." in main.go and save the file.
  • Notice how VS Code auto-formats your code on save (adding/removing imports, fixing indentation).
  • Change it back to "Hello, World!" and save again.

The Go development loop

Your typical workflow is: edit code in VS Code, save (auto-formats), run with go run main.go for quick feedback. Use go build when you want a deployable binary.

Your Go environment is ready and you've confirmed the full compile-and-run cycle works. Next up, you'll build the worker pool that checks URLs concurrently.

Build the Worker Pool with Goroutines and Channels

Your Go environment is ready and your module compiles. Now it's time to build the engine that makes your URL checker fast.

A worker pool is a set of goroutines that pull jobs from a shared channel and process them concurrently. Instead of checking URLs one by one, you'll have multiple workers grabbing URLs from a queue and checking them in parallel.

In this step, get ready to:
  • Define Job and Result structs to represent work items and their outcomes.
  • Write a worker function that makes HTTP HEAD requests and reports results.
  • Wire up main to dispatch jobs across workers and print a summary table.
Define the data structures

Every worker pool needs two things: a description of the work to do, and a place to put the results. In Go, you model these as structs.

  • In VS Code, open main.go and replace its entire contents with the following code:
package main

import (
	"fmt"
	"net"
	"net/http"
	"os"
	"strconv"
	"sync"
	"time"
)

// Job represents a single URL to check
type Job struct {
	ID  int
	URL string
}

// Result holds the outcome of checking one URL
type Result struct {
	Job        Job
	StatusCode int
	Duration   time.Duration
	Error      error
	Category   string
}

What do these structs do?

Job is a work item. Each Job carries an ID (for tracking) and a URL to check.

Result captures what happened when a worker checked a URL. It stores the HTTP status code, how long the request took, any error that occurred, and a category string that labels the outcome.

You won't see a change yet. The next chunk adds the worker function that uses these structs.

Implement the worker function

Each worker is a goroutine that loops over the jobs channel, makes an HTTP request for each job, and sends the result back on a separate channel.

  • Add the following worker function below the Result struct:
func worker(id int, jobs <-chan Job, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()
	for job := range jobs {
		fmt.Printf("[Worker %d] Checking %s\n", id, job.URL)
		start := time.Now()

		// Create an HTTP client with a 5-second timeout
		client := &http.Client{Timeout: 5 * time.Second}
		resp, err := client.Head(job.URL)
		duration := time.Since(start)

		if err != nil {
			results <- Result{
				Job:      job,
				Duration: duration,
				Error:    err,
				Category: "error",
			}
			continue
		}
		resp.Body.Close()

		results <- Result{
			Job:        job,
			StatusCode: resp.StatusCode,
			Duration:   duration,
			Category:   "ok",
		}
	}
}

What does this function do?

  • jobs <-chan Job means this channel is receive-only. The worker can only pull jobs out, not push them in.
  • results chan<- Result means this channel is send-only. The worker can only send results back.
  • defer wg.Done() tells the WaitGroup that this worker is finished when the function returns.
  • client.Head(job.URL) makes an HTTP HEAD request. HEAD is lighter than GET because it only fetches headers, not the full page body.
  • The continue keyword skips to the next job if an error occurs, so one failed URL doesn't stop the worker.

You won't see a change yet. The next chunk wires everything together in main().

Wire up main to dispatch jobs and collect results

The main function is the orchestrator. It creates channels, launches workers, feeds them jobs, and collects the results into a summary table.

  • Add the following main function below the worker function:
func main() {
	// URLs to check - mix of valid, invalid, and slow
	urls := []string{
		"https://go.dev",
		"https://github.com",
		"https://google.com",
		"https://httpbin.org/status/200",
		"https://httpbin.org/status/404",
		"https://thisdomaindoesnotexist12345.com",
		"https://httpbin.org/delay/3",
		"https://example.com",
	}

	// Default to 3 workers, or read from command line
	numWorkers := 3
	if len(os.Args) > 1 {
		if n, err := strconv.Atoi(os.Args[1]); err == nil && n > 0 {
			numWorkers = n
		}
	}

	fmt.Printf("URL Health Checker\n")
	fmt.Printf("Workers: %d | URLs: %d\n\n", numWorkers, len(urls))

	// Create buffered channels for jobs and results
	jobs := make(chan Job, len(urls))
	results := make(chan Result, len(urls))

	var wg sync.WaitGroup

	// Launch worker goroutines
	for w := 1; w <= numWorkers; w++ {
		wg.Add(1)
		go worker(w, jobs, results, &wg)
	}

	// Send all jobs into the channel
	for i, url := range urls {
		jobs <- Job{ID: i + 1, URL: url}
	}
	close(jobs)

	// Close results channel once all workers finish
	go func() {
		wg.Wait()
		close(results)
	}()

	// Print results as a table
	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}
	_ = net.DefaultResolver
}

What's happening here?

  • make(chan Job, len(urls)) creates a buffered channel. Buffered means you can send all jobs into it without blocking, even if no worker has started reading yet.
  • go worker(w, jobs, results, &wg) launches each worker as a goroutine. All 3 workers start immediately and race to pull jobs from the same channel.
  • close(jobs) signals to workers that no more jobs are coming. Their range jobs loop exits when the channel is closed and drained.
  • The anonymous goroutine waits for all workers to finish (wg.Wait()), then closes the results channel. This lets the range results loop in main know when all results have arrived.
  • _ = net.DefaultResolver keeps the net import alive. You'll use it properly in the next step for error classification.
  • Save main.go.
  • Run your program by executing this command in the terminal:
go run main.go

You should see interleaved log output from different worker IDs, followed by a summary table showing each URL's status, response time, and category.

What should I see?

The output starts with worker log lines like [Worker 1] Checking https://go.dev. These appear interleaved because all 3 workers run concurrently.

After all checks complete, a table prints each URL with its HTTP status code (or ERR for failures), response time, and category. The invalid domain shows ERR because DNS resolution fails.

Seeing a compile error?

  • If you see imported and not used, check that your import block only includes packages you actually reference in the code. The Go extension may auto-remove unused imports on save.
  • If you see undefined: Job or undefined: Result, make sure the struct definitions appear above the worker function in main.go. Go reads top to bottom.
  • If you see undeclared name: worker, make sure the worker function appears above main() in the file.

Help me debug my Go compile error

✔️ Awesome, I've got everything!

Great. Double check you've saved main.go and your output shows all 8 URLs with their statuses.

ⓧ I'd like to double check the full code

Here is the complete main.go at this point:

package main

import (
	"fmt"
	"net"
	"net/http"
	"os"
	"strconv"
	"sync"
	"time"
)

// Job represents a single URL to check
type Job struct {
	ID  int
	URL string
}

// Result holds the outcome of checking one URL
type Result struct {
	Job        Job
	StatusCode int
	Duration   time.Duration
	Error      error
	Category   string
}

func worker(id int, jobs <-chan Job, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()
	for job := range jobs {
		fmt.Printf("[Worker %d] Checking %s\n", id, job.URL)
		start := time.Now()

		// Create an HTTP client with a 5-second timeout
		client := &http.Client{Timeout: 5 * time.Second}
		resp, err := client.Head(job.URL)
		duration := time.Since(start)

		if err != nil {
			results <- Result{
				Job:      job,
				Duration: duration,
				Error:    err,
				Category: "error",
			}
			continue
		}
		resp.Body.Close()

		results <- Result{
			Job:        job,
			StatusCode: resp.StatusCode,
			Duration:   duration,
			Category:   "ok",
		}
	}
}

func main() {
	// URLs to check - mix of valid, invalid, and slow
	urls := []string{
		"https://go.dev",
		"https://github.com",
		"https://google.com",
		"https://httpbin.org/status/200",
		"https://httpbin.org/status/404",
		"https://thisdomaindoesnotexist12345.com",
		"https://httpbin.org/delay/3",
		"https://example.com",
	}

	// Default to 3 workers, or read from command line
	numWorkers := 3
	if len(os.Args) > 1 {
		if n, err := strconv.Atoi(os.Args[1]); err == nil && n > 0 {
			numWorkers = n
		}
	}

	fmt.Printf("URL Health Checker\n")
	fmt.Printf("Workers: %d | URLs: %d\n\n", numWorkers, len(urls))

	// Create buffered channels for jobs and results
	jobs := make(chan Job, len(urls))
	results := make(chan Result, len(urls))

	var wg sync.WaitGroup

	// Launch worker goroutines
	for w := 1; w <= numWorkers; w++ {
		wg.Add(1)
		go worker(w, jobs, results, &wg)
	}

	// Send all jobs into the channel
	for i, url := range urls {
		jobs <- Job{ID: i + 1, URL: url}
	}
	close(jobs)

	// Close results channel once all workers finish
	go func() {
		wg.Wait()
		close(results)
	}()

	// Print results as a table
	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}
	_ = net.DefaultResolver
}

Your worker pool is alive. Three goroutines are racing through 8 URLs simultaneously, and you can see the concurrency in the interleaved output. Next up, you'll add proper timeout handling with context and classify different types of errors so your output tells you exactly why a URL failed.

Add Timeouts and Error Handling with Context

Your worker pool is dispatching jobs and collecting results concurrently. Right now, though, every failure gets the same generic "error" label.

In production systems, you need to know why a request failed. A DNS error means the domain doesn't exist. A timeout means the server is slow. In this step, you'll switch to Go's context package for fine-grained timeout control and add error classification so each failure tells you exactly what went wrong.

In this step, get ready to:
  • Replace the simple HTTP client timeout with context-based request control.
  • Classify errors into distinct categories like timeout, DNS failure, and connection refused.
  • Verify that invalid URLs produce clear, actionable error labels.
Add context-based timeout to the HTTP client

Go's context package gives you a way to propagate deadlines, cancellation signals, and request-scoped values across API boundaries. Instead of setting a flat timeout on the HTTP client, you attach a context directly to each request. This means you can cancel individual requests without affecting the whole client.

  • In main.go, find the worker function.
  • Replace the entire function body (everything inside the for job := range jobs loop) with this updated version that uses context-based timeouts:
func worker(id int, jobs <-chan Job, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()
	for job := range jobs {
		fmt.Printf("[Worker %d] Checking %s\n", id, job.URL)
		start := time.Now()

		// Create a context with a 5-second deadline for this specific request
		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)

		// Build the request with context attached
		req, err := http.NewRequestWithContext(ctx, http.MethodHead, job.URL, nil)
		if err != nil {
			cancel()
			results <- Result{
				Job:      job,
				Duration: time.Since(start),
				Error:    err,
				Category: "request_error",
			}
			continue
		}

		// Execute the request with a plain client (context handles the timeout)
		client := &http.Client{}
		resp, err := client.Do(req)
		cancel()
		duration := time.Since(start)

		if err != nil {
			category := classifyError(err)
			results <- Result{
				Job:      job,
				Duration: duration,
				Error:    err,
				Category: category,
			}
			continue
		}
		resp.Body.Close()

		results <- Result{
			Job:        job,
			StatusCode: resp.StatusCode,
			Duration:   duration,
			Category:   "ok",
		}
	}
}

What does this code do?

The key change is how the timeout works. Instead of a blanket client-level timeout, each request gets its own context with a 5-second deadline.

  • context.WithTimeout(context.Background(), 5*time.Second) creates a fresh deadline for each URL check. If the request takes longer than 5 seconds, the context cancels it automatically.
  • http.NewRequestWithContext attaches the context to the HTTP request. The http.MethodHead constant replaces the previous Head() shortcut.
  • cancel() is called after the request completes (or fails) to release the context's resources. Forgetting to cancel leaks memory.
  • classifyError(err) replaces the generic "error" category. You'll write this function next.

Why context instead of a client timeout?

A client-level timeout applies to every request made with that client. Context gives you per-request control. In the Secret Mission later, you'll pass a parent context from main down to workers. When you press Ctrl+C, the parent context cancels, and every in-flight request cancels too. That propagation is only possible with context-based requests.

Your code won't compile yet because classifyError doesn't exist. You'll add it in the next substep.

Add error classification functions

Go treats errors as values, not exceptions. This means you can inspect, compare, and classify them. Your classifyError function will use type assertions to check whether an error is a timeout, a DNS failure, or a connection refusal.

  • Below the worker function's closing brace, add the classifyError function:
func classifyError(err error) string {
	// Check if the error is a network timeout
	if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
		return "timeout"
	}
	// Check if it's a DNS resolution failure
	var dnsErr *net.DNSError
	if ok := errorAs(err, &dnsErr); ok {
		return "dns_error"
	}
	// Check if the connection was refused
	var opErr *net.OpError
	if ok := errorAs(err, &opErr); ok {
		return "connection_refused"
	}
	return "unknown_error"
}

What does this code do?

  • err.(net.Error) is a type assertion. It checks if the error implements the net.Error interface, which has a Timeout() method.
  • *net.DNSError represents a DNS lookup failure. This fires for domains that don't exist.
  • *net.OpError represents a low-level network operation failure like a refused connection.
  • The function uses errorAs, a helper you'll add next that unwraps wrapped errors to find the underlying type.
  • Directly below classifyError, add the errorAs helper function:
func errorAs(err error, target interface{}) bool {
	// Use a type switch to handle different target pointer types
	switch t := target.(type) {
	case **net.DNSError:
		for {
			if e, ok := err.(*net.DNSError); ok {
				*t = e
				return true
			}
			// Try to unwrap the error one layer at a time
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	case **net.OpError:
		for {
			if e, ok := err.(*net.OpError); ok {
				*t = e
				return true
			}
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	}
	return false
}

Why a custom errorAs helper?

Go's standard library has an errors.As function, but it requires generics-aware usage in newer Go versions. This custom implementation makes the unwrapping process explicit and educational.

The function loops through wrapped error layers (using the Unwrap() interface) until it finds the specific error type or runs out of layers. This is the same unwrapping pattern the standard library uses internally.

  • Scroll to the very end of your main() function and find this line:
	_ = net.DefaultResolver
  • Delete this line. It was a placeholder to keep the net import from erroring. Your new classifyError and errorAs functions now use the net package directly.
  • Save main.go.
  • Run your program to verify the new error categories by running:
go run main.go

You should see the same summary table as before, but the CATEGORY column now shows specific labels. Valid URLs display ok. The fake domain thisdomaindoesnotexist12345.com shows dns_error. Slow URLs that exceed 5 seconds show timeout.

Seeing a compile error?

  • If you see "undefined: classifyError", make sure the classifyError function is defined outside of main() and outside of worker(). It should be a standalone function at the package level.
  • If you see "undefined: errorAs", confirm the errorAs function is also defined at the package level, below classifyError.
  • If you see "net imported and not used", make sure you deleted the _ = net.DefaultResolver line AND that your classifyError function references the net package types correctly.

Why won't my Go program compile after adding error classification?

Verify clean shutdown with WaitGroup

Your sync.WaitGroup pattern from the previous step is already handling goroutine lifecycle. Each worker calls wg.Done() when it finishes, and a background goroutine waits for all workers before closing the results channel.

This means your program exits cleanly with no goroutine leaks. Every worker finishes its in-flight request, reports its result, and the main function collects everything before terminating.

Why does clean shutdown matter?

A goroutine leak is when a goroutine stays alive after the program no longer needs it. In long-running services (web servers, background workers), leaked goroutines accumulate memory and file descriptors until the process crashes. The WaitGroup pattern guarantees every goroutine you start also finishes.

✔️ Awesome, I've got everything!

Great. Double check you've saved main.go before moving on.

ⓧ I'd like to double check the full code

Here's the complete main.go at this point:

package main

import (
	"context"
	"fmt"
	"net"
	"net/http"
	"os"
	"strconv"
	"sync"
	"time"
)

type Job struct {
	ID  int
	URL string
}

type Result struct {
	Job        Job
	StatusCode int
	Duration   time.Duration
	Error      error
	Category   string
}

func worker(id int, jobs <-chan Job, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()
	for job := range jobs {
		fmt.Printf("[Worker %d] Checking %s\n", id, job.URL)
		start := time.Now()

		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)

		req, err := http.NewRequestWithContext(ctx, http.MethodHead, job.URL, nil)
		if err != nil {
			cancel()
			results <- Result{
				Job:      job,
				Duration: time.Since(start),
				Error:    err,
				Category: "request_error",
			}
			continue
		}

		client := &http.Client{}
		resp, err := client.Do(req)
		cancel()
		duration := time.Since(start)

		if err != nil {
			category := classifyError(err)
			results <- Result{
				Job:      job,
				Duration: duration,
				Error:    err,
				Category: category,
			}
			continue
		}
		resp.Body.Close()

		results <- Result{
			Job:        job,
			StatusCode: resp.StatusCode,
			Duration:   duration,
			Category:   "ok",
		}
	}
}

func classifyError(err error) string {
	if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
		return "timeout"
	}
	var dnsErr *net.DNSError
	if ok := errorAs(err, &dnsErr); ok {
		return "dns_error"
	}
	var opErr *net.OpError
	if ok := errorAs(err, &opErr); ok {
		return "connection_refused"
	}
	return "unknown_error"
}

func errorAs(err error, target interface{}) bool {
	switch t := target.(type) {
	case **net.DNSError:
		for {
			if e, ok := err.(*net.DNSError); ok {
				*t = e
				return true
			}
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	case **net.OpError:
		for {
			if e, ok := err.(*net.OpError); ok {
				*t = e
				return true
			}
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	}
	return false
}

func main() {
	urls := []string{
		"https://go.dev",
		"https://github.com",
		"https://google.com",
		"https://httpbin.org/status/200",
		"https://httpbin.org/status/404",
		"https://thisdomaindoesnotexist12345.com",
		"https://httpbin.org/delay/3",
		"https://example.com",
	}

	numWorkers := 3
	if len(os.Args) > 1 {
		if n, err := strconv.Atoi(os.Args[1]); err == nil && n > 0 {
			numWorkers = n
		}
	}

	fmt.Printf("URL Health Checker\n")
	fmt.Printf("Workers: %d | URLs: %d\n\n", numWorkers, len(urls))

	jobs := make(chan Job, len(urls))
	results := make(chan Result, len(urls))

	var wg sync.WaitGroup

	for w := 1; w <= numWorkers; w++ {
		wg.Add(1)
		go worker(w, jobs, results, &wg)
	}

	for i, url := range urls {
		jobs <- Job{ID: i + 1, URL: url}
	}
	close(jobs)

	go func() {
		wg.Wait()
		close(results)
	}()

	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}
}

Your health checker now gives you actionable error reports instead of a vague "error" label. Next up, you'll add timing measurements to prove that your worker pool is actually faster than checking URLs one at a time.

Polish the Output and Measure Concurrency Gains

Your worker pool handles timeouts and classifies errors cleanly. But right now, you can't actually see how much faster concurrency makes things.

In this step, you'll add timing instrumentation that measures total elapsed time versus what sequential execution would have taken. Then you'll run the program with different worker counts to see the speedup in real numbers.

In this step, get ready to:
  • Add timing code that calculates concurrent vs sequential execution time.
  • Run the program with 1, 3, and 5 workers to observe measurable runtime differences.
Add timing to demonstrate concurrency benefit

To prove concurrency helps, you need two measurements: how long the program actually took (concurrent), and how long it would have taken if every request ran one after another (sequential). The sequential time is the sum of all individual response durations.

  • In main.go, scroll down to the results collection section in main().
  • Find these lines:
	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}
  • Replace that entire block (from the table header through the closing } of the for loop) with the following code:
	start := time.Now()
	var totalResponseTime time.Duration
	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		totalResponseTime += result.Duration
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}

	elapsed := time.Since(start)
	fmt.Println("-----------------------------------------------------------------------------------------------")
	fmt.Printf("\nTotal time (concurrent):   %s\n", elapsed.Round(time.Millisecond))
	fmt.Printf("Total time (sequential):   %s\n", totalResponseTime.Round(time.Millisecond))
	fmt.Printf("Speedup:                   %.1fx\n", float64(totalResponseTime)/float64(elapsed))

What does this code do?

This adds three pieces of timing instrumentation to your results collection.

  • start := time.Now() captures the wall-clock time right before results start flowing in.
  • totalResponseTime += result.Duration accumulates every individual request's duration. This represents what sequential execution would cost.
  • elapsed := time.Since(start) captures how long the concurrent execution actually took.
  • The speedup factor divides the sequential total by the concurrent total. A value of 2.5x means concurrency made your program 2.5 times faster than checking URLs one by one.
  • Save main.go.
  • Run your program to confirm the timing output by running this command:
go run main.go

After the URL table, you should see a summary with total concurrent time, total sequential time, and a speedup factor greater than 1.0x.

Not seeing the timing summary?

  • Make sure you replaced the entire original for loop block, not just added code below it. The totalResponseTime += result.Duration line must be inside the loop.
  • Check that start := time.Now() appears before the table header, not after the loop.
  • If you see a compile error about start being redeclared, make sure you removed the original for loop and its surrounding code before pasting the replacement.

Still stuck? Help me debug my timing summary code in main.go

✔️ Awesome, I've got everything!

Great. Double check you've saved main.go before moving on.

ⓧ I'd like to double check the full code

package main

import (
	"context"
	"fmt"
	"net"
	"net/http"
	"os"
	"strconv"
	"sync"
	"time"
)

type Job struct {
	ID  int
	URL string
}

type Result struct {
	Job        Job
	StatusCode int
	Duration   time.Duration
	Error      error
	Category   string
}

func worker(id int, jobs <-chan Job, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()
	for job := range jobs {
		fmt.Printf("[Worker %d] Checking %s\n", id, job.URL)
		start := time.Now()

		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)

		req, err := http.NewRequestWithContext(ctx, http.MethodHead, job.URL, nil)
		if err != nil {
			cancel()
			results <- Result{
				Job:      job,
				Duration: time.Since(start),
				Error:    err,
				Category: "request_error",
			}
			continue
		}

		client := &http.Client{}
		resp, err := client.Do(req)
		cancel()
		duration := time.Since(start)

		if err != nil {
			category := classifyError(err)
			results <- Result{
				Job:      job,
				Duration: duration,
				Error:    err,
				Category: category,
			}
			continue
		}
		resp.Body.Close()

		results <- Result{
			Job:        job,
			StatusCode: resp.StatusCode,
			Duration:   duration,
			Category:   "ok",
		}
	}
}

func classifyError(err error) string {
	if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
		return "timeout"
	}
	var dnsErr *net.DNSError
	if ok := errorAs(err, &dnsErr); ok {
		return "dns_error"
	}
	var opErr *net.OpError
	if ok := errorAs(err, &opErr); ok {
		return "connection_refused"
	}
	return "unknown_error"
}

func errorAs(err error, target interface{}) bool {
	switch t := target.(type) {
	case **net.DNSError:
		for {
			if e, ok := err.(*net.DNSError); ok {
				*t = e
				return true
			}
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	case **net.OpError:
		for {
			if e, ok := err.(*net.OpError); ok {
				*t = e
				return true
			}
			if unwrap, ok := err.(interface{ Unwrap() error }); ok {
				err = unwrap.Unwrap()
			} else {
				return false
			}
		}
	}
	return false
}

func main() {
	urls := []string{
		"https://go.dev",
		"https://github.com",
		"https://google.com",
		"https://httpbin.org/status/200",
		"https://httpbin.org/status/404",
		"https://thisdomaindoesnotexist12345.com",
		"https://httpbin.org/delay/3",
		"https://example.com",
	}

	numWorkers := 3
	if len(os.Args) > 1 {
		if n, err := strconv.Atoi(os.Args[1]); err == nil && n > 0 {
			numWorkers = n
		}
	}

	fmt.Printf("URL Health Checker\n")
	fmt.Printf("Workers: %d | URLs: %d\n\n", numWorkers, len(urls))

	jobs := make(chan Job, len(urls))
	results := make(chan Result, len(urls))

	var wg sync.WaitGroup

	for w := 1; w <= numWorkers; w++ {
		wg.Add(1)
		go worker(w, jobs, results, &wg)
	}

	for i, url := range urls {
		jobs <- Job{ID: i + 1, URL: url}
	}
	close(jobs)

	go func() {
		wg.Wait()
		close(results)
	}()

	start := time.Now()
	var totalResponseTime time.Duration
	fmt.Printf("\n%-45s %-12s %-10s %s\n", "URL", "STATUS", "TIME", "CATEGORY")
	fmt.Println("-----------------------------------------------------------------------------------------------")

	for result := range results {
		totalResponseTime += result.Duration
		status := fmt.Sprintf("%d", result.StatusCode)
		if result.Error != nil {
			status = "ERR"
		}
		fmt.Printf("%-45s %-12s %-10s %s\n",
			result.Job.URL,
			status,
			result.Duration.Round(time.Millisecond),
			result.Category,
		)
	}

	elapsed := time.Since(start)
	fmt.Println("-----------------------------------------------------------------------------------------------")
	fmt.Printf("\nTotal time (concurrent):   %s\n", elapsed.Round(time.Millisecond))
	fmt.Printf("Total time (sequential):   %s\n", totalResponseTime.Round(time.Millisecond))
	fmt.Printf("Speedup:                   %.1fx\n", float64(totalResponseTime)/float64(elapsed))
}
Run with different worker counts to observe concurrency gains

Your program already accepts an optional worker count as a command-line argument. Passing a number after main.go sets the number of concurrent workers. Now you can see exactly how parallelism affects total runtime.

  • Run with a single worker to simulate sequential execution:
go run main.go 1

Notice the total concurrent time is roughly equal to the sequential time. With only one worker, there is no parallelism.

  • Now run with the default 3 workers:
go run main.go 3

The concurrent time drops significantly. The speedup factor should be around 2-3x.

  • Finally, try with 5 workers:
go run main.go 5

With 5 workers for 8 URLs, most requests run in parallel. You should see a higher speedup factor than the 3-worker run.

Why doesn't 5 workers mean 5x faster?

Speedup depends on real-world factors like network latency, DNS resolution time, and server response speed. Some URLs respond instantly while others (like the intentional 3-second delay) create a bottleneck regardless of worker count.

This is why the speedup is never a perfect multiple of the worker count. In production systems, finding the optimal worker count for a given workload is a tuning exercise.

  • Compile the final binary by running this command:
go build
  • Confirm the binary exists by running:
ls url-checker

You should see url-checker listed. This is your compiled binary that you can run directly with ./url-checker or ./url-checker 5.

Seeing 1.0x speedup with multiple workers?

  • Check that start := time.Now() is placed before the table header print, not inside the for loop.
  • Make sure your elapsed := time.Since(start) is after the for loop ends, not inside it.
  • If your network is very fast, the difference between 1 and 3 workers might be small. Try adding more URLs or using slower endpoints.

Still stuck? Help me understand why my speedup factor is always 1.0x

Your URL health checker is complete. You have a working concurrent Go program that proves goroutines and channels deliver real performance gains. Next up, the Secret Mission challenges you to add graceful shutdown so pressing Ctrl+C drains in-flight workers cleanly.

Secret mission

Add Graceful Shutdown with OS Signal Handling

Your health checker works great, but what happens when you press Ctrl+C mid-execution? Right now, the program dies instantly and you lose all results. In production Go services, graceful shutdown is non-negotiable. Can you wire up OS signal handling so Ctrl+C cleanly drains in-flight workers and prints partial results?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your project files or delete them.

Resources you used:

  • A Go module directory (url-checker/) on your Desktop.
  • A compiled binary (url-checker) inside that directory.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with Go concurrency patterns.

  • Your url-checker/ folder stays on your Desktop. There is no ongoing cost.
  • Go remains installed at /usr/local/go and is available for future projects.
  • The VS Code Go extension stays active for any Go files you open.

Pause - I'll come back to this later

This project has no running processes or cloud resources to pause. The CLI tool runs and exits immediately.

  • Your project files remain on disk, ready to pick up where you left off.
  • To resume, open the url-checker/ folder in VS Code and run go run main.go again.

Delete - I don't want to use this again

Remove all project files from your machine. This deletes the source code and compiled binary permanently.

  • Delete the project folder by running this command in your terminal:
rm -rf ~/Desktop/url-checker
  • Confirm the folder is gone by running:
ls ~/Desktop/url-checker
  • You should see a "No such file or directory" message.
  • Go itself remains installed. You can keep it for future projects or remove it by deleting /usr/local/go and removing the Go path entry from your shell profile.

Nice Work!

Nice Work!

You just built a concurrent URL health checker from scratch in Go. That single file packs goroutines, channels, a worker pool, context-based timeouts, and structured error handling into a tool you can run from your terminal right now.

You've learned how to:

  • Implement a worker pool using goroutines and channels to process jobs concurrently across multiple workers.
  • Handle timeouts and classify errors using context, the Go-idiomatic approach to cancellation and deadline propagation.
  • Manage goroutine lifecycles with sync.WaitGroup and measure real concurrency speedup by comparing parallel vs sequential execution time.
  • Secret Mission: Implemented graceful shutdown with OS signal handling so pressing Ctrl+C cleanly drains in-flight workers before exiting.

Ready to quiz yourself?