Build a Distributed File Storage System

Build a distributed file storage system with chunking and integrity checks.

Introduction

30 Second Summary

When you upload a photo to the cloud, it doesn't sit on a single hard drive somewhere. It gets split into pieces and spread across multiple machines so that even if one breaks, your file is safe.

In this project, you will build a distributed file storage system from scratch using Node.js. You'll create three independent services that split files into chunks, distribute them across storage nodes, and reassemble them on download.

What You'll Build

A multi-service distributed storage system with a coordinator for metadata, multiple storage nodes for data persistence, and a CLI client for uploading and downloading files over HTTP.

By the end of this project, you'll have:

  • A Coordinator service that manages file metadata and maps chunks to storage nodes.
  • Multiple Storage Node services that persist file chunks to disk using streaming I/O.
  • A Client CLI that splits files into chunks, distributes them with round-robin allocation, and verifies integrity using SHA-256 hashes.
  • Secret Mission: Add chunk replication so each chunk is stored on two nodes with automatic failover on download.

Are there any prerequisites?

None. This guide includes setup for all required tools. You just need a computer with a code editor and an internet connection to download Node.js.

Before We Start

To build our distributed file storage system, we need a JavaScript runtime that can create HTTP servers and handle file I/O. Node.js gives us all of this out of the box with zero external dependencies.

In this step, you will install Node.js, create the project folder structure, and set up a test file for later uploads.

In this step, get ready to:
  • Install Node.js v24 LTS.
  • Create the project directory with three service files.
  • Create a test file for upload testing.
Download and install Node.js v24 LTS

Node.js is the JavaScript runtime that powers all three services in our distributed system. You need version 24 or higher for this project.

  • Open the Node.js download page in your browser:
  • https://nodejs.org/en/download
  • Click the Windows Installer (.msi) option for 64-bit.
  • Run the downloaded node-v24.17.0-x64.msi file.
  • Follow the installer prompts, accepting the default settings at each screen.
  • Click Finish when the installation completes.

What does the installer do?

The MSI installer adds the node and npm commands to your system PATH.

This means you can run JavaScript files and install packages from any terminal window after the install completes.

Now verify the installation works.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.
  • Check your Node.js version by running this command:
node -v
  • Check your npm version by running:
npm -v

✔️ I see a version starting with v24

Node.js is installed and ready. You should see something like v24.17.0 for the first command and a version number like 11.13.0 for the second.

ⓧ I see an older version

You have a previous version of Node.js installed. You need v24 or higher for this project.

  • Go to https://nodejs.org/en/download and download the latest v24 LTS Windows Installer.
  • Run the MSI installer. It will upgrade your existing installation.
  • Close your current PowerShell window and open a new one.
  • Run node -v again to confirm it now shows v24 or higher.

ⓧ 'node' is not recognized

Your terminal cannot find the Node.js executable. This usually means the installer did not complete or the terminal was already open before installation.

  • Close your current PowerShell window completely.
  • Open a fresh PowerShell window (press the Windows key, type PowerShell, press Enter).
  • Run node -v again.
  • If it still fails, re-run the MSI installer from the download page and make sure the Add to PATH option is checked during setup.
Create the project directory and file structure

Our distributed system has three independent services, each in its own file. You will create the project folder and the three empty files now.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir distributed-storage
cd distributed-storage

Now create the three service files. You will do this in your code editor.

  • Open the distributed-storage folder in your code editor.
  • Create a new file called storage-node.js in the root of the folder.
  • Create a new file called coordinator.js in the root of the folder.
  • Create a new file called client.js in the root of the folder.
  • Leave all three files empty for now.
  • Confirm the files exist by running:
ls

You should see client.js, coordinator.js, and storage-node.js listed in the output.

Don't see your files?

  • Make sure your terminal is inside the distributed-storage folder. Run pwd to check your current location.
  • Confirm you saved each file in the correct folder (not a subfolder or your Desktop).

Still stuck? Help me find where my files ended up.

Create a test file for later use

You need a sample file with enough content to produce multiple chunks when uploaded. A file of a few kilobytes is ideal for local testing.

  • Create the test-files directory and generate a sample file by running this command:
mkdir test-files
node -e "const fs = require('fs'); fs.writeFileSync('./test-files/sample.txt', Array(100).fill('The quick brown fox jumps over the lazy dog. This is sample content for testing our distributed file storage system.\n').join(''))"

What does this command do?

The first line creates a test-files directory inside your project folder.

The second line uses Node.js to generate a text file with 100 repeated lines. This produces roughly 11 KB of content, enough to split into multiple chunks during upload.

  • Verify the file was created by running:
ls test-files

You should see sample.txt listed in the output.

Seeing an error about 'fs' or 'require'?

  • Make sure you completed the Node.js installation in the previous substep. The node -e command needs a working Node.js installation.
  • If you see a syntax error, check that you copied the entire command including the closing quote.

Need more help? Help me troubleshoot this Node.js command.

Your environment is ready and your project structure is in place. Next up, you will build the first service: the Storage Node that persists file chunks to disk.

Build the Storage Node

Your project structure is ready. Now it's time to build the first service in your distributed system: the storage node.

In production systems like Google File System and HDFS, storage nodes (called "ChunkServers" or "DataNodes") are the machines that physically hold your file data. Your storage node will accept file chunks over HTTP and persist them to disk using streaming I/O, so it can handle files of any size without running out of memory.

In this step, get ready to:
  • Create an HTTP server that accepts a port and storage directory as command-line arguments.
  • Implement streaming upload and download routes for file chunks.
  • Test the storage node by verifying its health check endpoint.
Set up server configuration and the storage directory

Before routing any requests, the storage node needs to know which port to listen on and where to save chunks on disk. You will pass both values as command-line arguments so you can run multiple nodes side by side.

  • Open storage-node.js in your code editor and add the following code:
const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

// Read port and storage directory from command-line arguments
const PORT = parseInt(process.argv[2]) || 4001;
const STORAGE_DIR = process.argv[3] || `./storage-data-${PORT}`;

// Create the storage directory if it doesn't exist yet
if (!fs.existsSync(STORAGE_DIR)) {
  fs.mkdirSync(STORAGE_DIR, { recursive: true });
}

What does this code do?

The first three lines import Node.js built-in modules for HTTP servers, file system operations, and path handling.

PORT and STORAGE_DIR read from process.argv (the command-line arguments). If none are provided, they default to port 4001 and a folder named after the port. The mkdirSync call with { recursive: true } creates the storage folder (and any parent folders) if it doesn't already exist.

  • Save storage-node.js.
Implement the streaming upload route

The upload route streams incoming request data directly to a file on disk. This means the node never loads an entire chunk into memory, so it can handle chunks of any size.

  • Add the following code below the mkdirSync block in storage-node.js:
const server = http.createServer((req, res) => {
  const parts = req.url.split('/');

  // PUT /chunks/:id - Store a chunk by streaming request body to disk
  if (req.method === 'PUT' && parts[1] === 'chunks' && parts[2]) {
    const chunkId = parts[2];
    const filePath = path.join(STORAGE_DIR, chunkId);
    const writeStream = fs.createWriteStream(filePath);

    req.pipe(writeStream);

    writeStream.on('finish', () => {
      res.writeHead(201, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ status: 'stored', chunkId }));
    });

    writeStream.on('error', (err) => {
      res.writeHead(500, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    });

    return;
  }
});

What does this code do?

http.createServer creates a new HTTP server. Every incoming request hits the callback function, where req.url.split('/') breaks the URL path into segments for routing.

The key line is req.pipe(writeStream). This connects the incoming request stream directly to a file write stream. Data flows from the network to disk in small pieces without ever buffering the full chunk in memory.

When the stream finishes, the server responds with a 201 status. If something goes wrong writing to disk, it responds with a 500 error.

Why streaming instead of buffering?

Without streaming, you would collect the entire request body into a variable before writing it. That works for small files, but a 500MB chunk would consume 500MB of memory.

Streaming processes data in small pieces as it arrives. Your storage node could handle gigabyte-sized chunks while using only a few kilobytes of memory.

  • Save storage-node.js.

Seeing a syntax error?

Make sure the closing }); at the bottom closes both the if block and the createServer callback. You will add more routes inside this callback in the next chunk.

help me find the syntax error in my storage-node.js

Implement the download and health check routes

The download route reverses the upload process. Instead of streaming from network to disk, it streams from disk to the network. The health route lets other services confirm this node is running.

  • In storage-node.js, find the closing }); of your createServer callback.
  • Replace that }); with the following code (which adds the remaining routes and then closes the server callback):
  // GET /chunks/:id - Retrieve a chunk by streaming from disk to response
  if (req.method === 'GET' && parts[1] === 'chunks' && parts[2]) {
    const chunkId = parts[2];
    const filePath = path.join(STORAGE_DIR, chunkId);

    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: 'Chunk not found' }));
      return;
    }

    const stat = fs.statSync(filePath);
    res.writeHead(200, {
      'Content-Type': 'application/octet-stream',
      'Content-Length': stat.size,
    });

    const readStream = fs.createReadStream(filePath);
    readStream.pipe(res);

    return;
  }

  // GET /health - Health check for other services to verify this node is alive
  if (req.method === 'GET' && parts[1] === 'health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'healthy', port: PORT }));
    return;
  }

  // Fallback for unknown routes
  res.writeHead(404, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ error: 'Not found' }));
});

server.listen(PORT, () => {
  console.log(`Storage Node running on port ${PORT}`);
  console.log(`Storing chunks in: ${path.resolve(STORAGE_DIR)}`);
});

What does this code do?

The GET /chunks/:id route uses readStream.pipe(res) to stream a file from disk directly into the HTTP response. This is the mirror image of the upload route. It also sets the Content-Length header so the client knows exactly how many bytes to expect.

The /health route returns a simple JSON object confirming the node is alive. The coordinator will use this to check node availability.

Finally, server.listen(PORT) starts the server and prints a startup message with the resolved storage directory path.

  • Save storage-node.js.

Seeing unexpected token errors?

Make sure you removed the old }); before pasting. You should have exactly one }); that closes the createServer callback, followed by the server.listen call.

help me fix the syntax in my storage-node.js routes

✔️ Awesome, I've got everything!

Great. Double check you have saved storage-node.js before moving on.

ⓧ I'd like to double check the full code

Here is the complete storage-node.js file. Make sure yours matches exactly:

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const PORT = parseInt(process.argv[2]) || 4001;
const STORAGE_DIR = process.argv[3] || `./storage-data-${PORT}`;

if (!fs.existsSync(STORAGE_DIR)) {
  fs.mkdirSync(STORAGE_DIR, { recursive: true });
}

const server = http.createServer((req, res) => {
  const parts = req.url.split('/');

  if (req.method === 'PUT' && parts[1] === 'chunks' && parts[2]) {
    const chunkId = parts[2];
    const filePath = path.join(STORAGE_DIR, chunkId);
    const writeStream = fs.createWriteStream(filePath);

    req.pipe(writeStream);

    writeStream.on('finish', () => {
      res.writeHead(201, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ status: 'stored', chunkId }));
    });

    writeStream.on('error', (err) => {
      res.writeHead(500, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    });

    return;
  }

  if (req.method === 'GET' && parts[1] === 'chunks' && parts[2]) {
    const chunkId = parts[2];
    const filePath = path.join(STORAGE_DIR, chunkId);

    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: 'Chunk not found' }));
      return;
    }

    const stat = fs.statSync(filePath);
    res.writeHead(200, {
      'Content-Type': 'application/octet-stream',
      'Content-Length': stat.size,
    });

    const readStream = fs.createReadStream(filePath);
    readStream.pipe(res);

    return;
  }

  if (req.method === 'GET' && parts[1] === 'health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'healthy', port: PORT }));
    return;
  }

  res.writeHead(404, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ error: 'Not found' }));
});

server.listen(PORT, () => {
  console.log(`Storage Node running on port ${PORT}`);
  console.log(`Storing chunks in: ${path.resolve(STORAGE_DIR)}`);
});
Test the storage node

Time to see it in action. You will start the storage node on port 4001 and then verify it responds to health checks.

  • Open a terminal in your distributed-storage folder.
  • Start the storage node by running this command:
node storage-node.js 4001 ./storage-data-1

You should see output confirming the node is running:

Storage Node running on port 4001
Storing chunks in: C:\Users\...\distributed-storage\storage-data-1

What do the arguments mean?

4001 tells the server which port to listen on. ./storage-data-1 tells it where to save chunk files. Later you will run a second node on port 4002 with a different storage directory, giving you two independent storage locations.

  • Open a second terminal window (keep the first one running).
  • Verify the health endpoint by opening this URL in your browser:
http://localhost:4001/health

You should see a JSON response in your browser:

{"status":"healthy","port":4001}

Use curl.exe from a second terminal

If you prefer to stay in the terminal instead of opening a browser, run this in your second terminal:

curl.exe http://localhost:4001/health

On Windows, use curl.exe (not curl) because PowerShell aliases curl to a different command.

  • Once you have confirmed the health check works, press Ctrl+C in the first terminal to stop the storage node.

Not seeing any response?

Make sure the first terminal is still running the storage node. If it crashed, check for syntax errors in storage-node.js and compare with the full code tab above.

If your browser shows "This site can't be reached," confirm the port number matches. The URL must be http://localhost:4001/health (not https).

help me troubleshoot why my storage node isn't responding to health checks

Your storage node is fully operational. Next up, you will build the coordinator service that decides which storage node each chunk goes to.

Build the Coordinator

Your storage nodes can persist and retrieve chunks, but they have no idea which file those chunks belong to or where to find them.

In a distributed file system, a separate coordinator service solves this problem. It tracks file metadata (which chunks make up a file and where each chunk lives) without storing any file data itself. This is the same metadata/data plane separation used by Google File System, HDFS, and Amazon S3.

In this step, get ready to:
  • Create an HTTP server that accepts storage node URLs as arguments.
  • Implement routes for allocating chunks and registering file metadata.
  • Test the round-robin allocation strategy.
Create the server skeleton and configuration

The coordinator needs to know which storage nodes exist and how to distribute work across them. You will set this up using command-line arguments and a round-robin counter.

  • Open coordinator.js in your code editor and add the following configuration and helper function:
const http = require('node:http');

const PORT = parseInt(process.argv[2]) || 4000;

// In-memory registry to track all stored files
const fileRegistry = new Map();
// Storage node URLs passed as command-line arguments
const storageNodes = process.argv.slice(3);

// Counter for round-robin distribution
let nextNodeIndex = 0;

// Returns the next storage node URL, cycling through the list
function getNextNode() {
  const node = storageNodes[nextNodeIndex];
  nextNodeIndex = (nextNodeIndex + 1) % storageNodes.length;
  return node;
}

What does this code do?

The coordinator reads its port from the first argument and storage node URLs from any remaining arguments. This means you can register as many nodes as you want by adding more URLs when you start the coordinator.

The fileRegistry Map stores file metadata in memory. The getNextNode function implements round-robin allocation by cycling a counter through the storage nodes array.

  • Below the getNextNode function, add the parseBody helper that reads JSON from incoming requests:
// Reads and parses the JSON body from an incoming request
function parseBody(req) {
  return new Promise((resolve, reject) => {
    const chunks = [];
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      try {
        const body = Buffer.concat(chunks).toString();
        resolve(body ? JSON.parse(body) : {});
      } catch (err) {
        reject(err);
      }
    });
    req.on('error', reject);
  });
}

Why do you need parseBody?

Node.js HTTP requests arrive as streams. You cannot access the full body immediately. This helper collects all data chunks, concatenates them into a string, and parses the result as JSON. Wrapping it in a Promise lets you use await for clean async flow.

  • Below parseBody, add the server skeleton with a health check route and a 404 fallback:
const server = http.createServer(async (req, res) => {
  const parts = req.url.split('/');

  if (req.method === 'GET' && parts[1] === 'health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(
      JSON.stringify({
        status: 'healthy',
        registeredFiles: fileRegistry.size,
        storageNodes,
      })
    );
    return;
  }

  res.writeHead(404, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ error: 'Not found' }));
});

server.listen(PORT, () => {
  console.log(`Coordinator running on port ${PORT}`);
  console.log(`Registered storage nodes: ${storageNodes.join(', ')}`);
});

What does this code do?

The server callback is async so you can await parseBody() inside route handlers. The health endpoint reports how many files are registered and which storage nodes the coordinator knows about.

  • Save coordinator.js.

Seeing a syntax error?

Make sure the template literal in console.log uses backticks (`), not regular quotes. Also check that parseBody is defined above the server variable so it is available when routes call it.

Need more help? Debug my coordinator skeleton

Implement the coordinator routes

The coordinator exposes four routes that the client will call during upload and download. Two handle writes (allocating nodes and registering files) and two handle reads (listing files and fetching metadata).

  • In coordinator.js, find the line const parts = req.url.split('/'); inside the server callback. Add the POST /allocate and POST /files route handlers directly below it:
  if (req.method === 'POST' && parts[1] === 'allocate') {
    try {
      const body = await parseBody(req);
      const { chunkCount } = body;
      const allocations = [];

      // Assign a storage node for each chunk using round-robin
      for (let i = 0; i < chunkCount; i++) {
        allocations.push(getNextNode());
      }

      res.writeHead(200, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ allocations }));
    } catch (err) {
      res.writeHead(400, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    }
    return;
  }

  if (req.method === 'POST' && parts[1] === 'files') {
    try {
      const body = await parseBody(req);
      const { filename, totalSize, chunks } = body;

      // Store the file's metadata in the registry
      fileRegistry.set(filename, {
        filename,
        totalSize,
        chunks,
        uploadedAt: new Date().toISOString(),
      });

      res.writeHead(201, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ status: 'registered', filename }));
    } catch (err) {
      res.writeHead(400, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    }
    return;
  }

What do these routes do?

  • POST /allocate accepts a chunkCount and returns an array of storage node URLs. Each call to getNextNode() advances the round-robin counter.
  • POST /files stores the complete metadata for an uploaded file (filename, size, chunk list) in the fileRegistry Map. This is called after all chunks are stored on the nodes.
  • Below the POST /files handler (and above the GET /health check), add the two read routes:
  if (req.method === 'GET' && parts[1] === 'files' && !parts[2]) {
    // List all stored files with summary info
    const files = Array.from(fileRegistry.values()).map((f) => ({
      filename: f.filename,
      totalSize: f.totalSize,
      chunkCount: f.chunks.length,
      uploadedAt: f.uploadedAt,
    }));

    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ files }));
    return;
  }

  if (req.method === 'GET' && parts[1] === 'files' && parts[2]) {
    // Return full metadata for a specific file
    const filename = decodeURIComponent(parts[2]);
    const file = fileRegistry.get(filename);

    if (!file) {
      res.writeHead(404, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: 'File not found' }));
      return;
    }

    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify(file));
    return;
  }

What do these routes do?

  • GET /files returns a summary of every file in the registry (name, size, chunk count, upload time). The client uses this for its list command.
  • GET /files/:filename returns full metadata for one file, including the chunk locations. The client uses this during download to know where to fetch each chunk.
  • Save coordinator.js.

Seeing unexpected token errors?

Check that each route handler block ends with return; and has matching curly braces. The route handlers should appear between const parts = req.url.split('/'); and the GET /health block.

Still stuck? Help me fix my coordinator routes

✔️ Awesome, I've got everything!

Great. Make sure you have saved coordinator.js before moving on to testing.

ⓧ I'd like to double check the full code

Here is the complete coordinator.js file. Compare it against yours:

const http = require('node:http');

const PORT = parseInt(process.argv[2]) || 4000;

const fileRegistry = new Map();
const storageNodes = process.argv.slice(3);

let nextNodeIndex = 0;

function getNextNode() {
  const node = storageNodes[nextNodeIndex];
  nextNodeIndex = (nextNodeIndex + 1) % storageNodes.length;
  return node;
}

function parseBody(req) {
  return new Promise((resolve, reject) => {
    const chunks = [];
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      try {
        const body = Buffer.concat(chunks).toString();
        resolve(body ? JSON.parse(body) : {});
      } catch (err) {
        reject(err);
      }
    });
    req.on('error', reject);
  });
}

const server = http.createServer(async (req, res) => {
  const parts = req.url.split('/');

  if (req.method === 'POST' && parts[1] === 'allocate') {
    try {
      const body = await parseBody(req);
      const { chunkCount } = body;
      const allocations = [];

      for (let i = 0; i < chunkCount; i++) {
        allocations.push(getNextNode());
      }

      res.writeHead(200, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ allocations }));
    } catch (err) {
      res.writeHead(400, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    }
    return;
  }

  if (req.method === 'POST' && parts[1] === 'files') {
    try {
      const body = await parseBody(req);
      const { filename, totalSize, chunks } = body;

      fileRegistry.set(filename, {
        filename,
        totalSize,
        chunks,
        uploadedAt: new Date().toISOString(),
      });

      res.writeHead(201, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ status: 'registered', filename }));
    } catch (err) {
      res.writeHead(400, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: err.message }));
    }
    return;
  }

  if (req.method === 'GET' && parts[1] === 'files' && !parts[2]) {
    const files = Array.from(fileRegistry.values()).map((f) => ({
      filename: f.filename,
      totalSize: f.totalSize,
      chunkCount: f.chunks.length,
      uploadedAt: f.uploadedAt,
    }));

    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ files }));
    return;
  }

  if (req.method === 'GET' && parts[1] === 'files' && parts[2]) {
    const filename = decodeURIComponent(parts[2]);
    const file = fileRegistry.get(filename);

    if (!file) {
      res.writeHead(404, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ error: 'File not found' }));
      return;
    }

    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify(file));
    return;
  }

  if (req.method === 'GET' && parts[1] === 'health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(
      JSON.stringify({
        status: 'healthy',
        registeredFiles: fileRegistry.size,
        storageNodes,
      })
    );
    return;
  }

  res.writeHead(404, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ error: 'Not found' }));
});

server.listen(PORT, () => {
  console.log(`Coordinator running on port ${PORT}`);
  console.log(`Registered storage nodes: ${storageNodes.join(', ')}`);
});
Test the round-robin allocation

The round-robin strategy ensures chunks are spread evenly across nodes. Each time the coordinator allocates a chunk, it picks the next node in the list and wraps back to the first node after reaching the end.

  • Start the coordinator with two simulated storage nodes by running this command:
node coordinator.js 4000 http://localhost:4001 http://localhost:4002

You should see output confirming the coordinator is running on port 4000 and listing both storage node URLs.

  • Open a second terminal window. Send a test allocation request by running:
Invoke-WebRequest -Uri "http://localhost:4000/allocate" -Method Post -Body '{"chunkCount": 4}' -ContentType "application/json" | Select-Object -ExpandProperty Content

You should see a JSON response with an allocations array that alternates between the two node URLs. The pattern repeats because the counter wraps around.

Why round-robin?

Round-robin distributes chunks evenly without needing to monitor disk usage or network load. For a two-node cluster with four chunks, the allocation is: node 1, node 2, node 1, node 2. Each node gets exactly half the data.

Production systems like HDFS use more sophisticated strategies (rack awareness, disk utilization) but round-robin is the foundation they build on.

  • Stop the coordinator by pressing Ctrl+C in the first terminal.

Getting a connection refused error?

Make sure the coordinator is still running in the first terminal before sending the request from the second terminal. If you see a port-in-use error, another process may be using port 4000. Close it or choose a different port number.

Need more help? Help me test my coordinator

Your coordinator can now manage file metadata and distribute work across storage nodes. Next up, you will build the client CLI that ties the coordinator and storage nodes together into a working file upload and download flow.

Build the Client CLI

You have a coordinator that knows where chunks should go and storage nodes ready to persist data. The missing piece is a client that ties the system together.

In this step, you will build the CLI tool that splits files into chunks, distributes them across storage nodes, and reassembles them on download. This is the component that users interact with directly.

In this step, get ready to:
  • Write HTTP helper functions for communicating with the coordinator and storage nodes.
  • Implement the upload command with file chunking, SHA-256 hashing, and distributed storage.
  • Implement the download and list commands with integrity verification.
Add the HTTP helpers and upload command

The client needs to talk to two different services: the coordinator (for allocation and metadata) and the storage nodes (for chunk data). You will build reusable HTTP helper functions first, then the upload command that uses them.

  • Open client.js in your code editor.
  • Add the imports, constants, and the general-purpose HTTP request helper by pasting the following code:
const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');
const crypto = require('node:crypto');

const COORDINATOR_URL = 'http://localhost:4000';
const CHUNK_SIZE = 1024 * 1024;

function httpRequest(url, options = {}) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(url);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname + parsedUrl.search,
      method: options.method || 'GET',
      headers: options.headers || {},
    };

    const req = http.request(reqOptions, (res) => {
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => {
        const body = Buffer.concat(chunks);
        resolve({ statusCode: res.statusCode, body });
      });
    });

    req.on('error', reject);

    if (options.body) {
      req.write(options.body);
    }

    req.end();
  });
}

What does this code do?

COORDINATOR_URL points at the coordinator's address. CHUNK_SIZE sets each chunk to 1 MB (1024 * 1024 bytes).

The httpRequest function wraps Node's low-level http.request into a Promise. It parses the URL, sends the request, collects the response body, and resolves with the status code and body as a Buffer.

  • Below the httpRequest function, add the sendChunk helper that sends a single chunk to a storage node:
function sendChunk(nodeUrl, chunkId, chunkData) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(`${nodeUrl}/chunks/${chunkId}`);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname,
      method: 'PUT',
      headers: { 'Content-Length': chunkData.length },
    };

    const req = http.request(reqOptions, (res) => {
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => {
        const body = Buffer.concat(chunks).toString();
        resolve(JSON.parse(body));
      });
    });

    req.on('error', reject);
    req.write(chunkData);
    req.end();
  });
}

What does sendChunk do?

sendChunk sends a PUT request to a specific storage node at /chunks/:id. It writes the raw chunk data into the request body and waits for the storage node's confirmation response.

  • Below the sendChunk function, add the full upload command function:
async function upload(filePath) {
  const fullPath = path.resolve(filePath);

  if (!fs.existsSync(fullPath)) {
    console.error(`File not found: ${fullPath}`);
    process.exit(1);
  }

  const fileData = fs.readFileSync(fullPath);
  const filename = path.basename(fullPath);
  const totalSize = fileData.length;

  console.log(`Uploading: ${filename} (${totalSize} bytes)`);

  // Split the file into 1MB chunks
  const chunkDataList = [];
  for (let i = 0; i < fileData.length; i += CHUNK_SIZE) {
    chunkDataList.push(fileData.slice(i, i + CHUNK_SIZE));
  }

  console.log(`Split into ${chunkDataList.length} chunk(s)`);

  // Ask the coordinator which nodes to use
  const allocRes = await httpRequest(`${COORDINATOR_URL}/allocate`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ chunkCount: chunkDataList.length }),
  });

  const { allocations } = JSON.parse(allocRes.body.toString());

  // Send each chunk to its assigned storage node
  const chunks = [];
  for (let i = 0; i < chunkDataList.length; i++) {
    const chunkData = chunkDataList[i];
    const hash = crypto.createHash('sha256').update(chunkData).digest('hex');
    const chunkId = `${filename}-chunk-${i}-${hash.slice(0, 8)}`;
    const nodeUrl = allocations[i];

    await sendChunk(nodeUrl, chunkId, chunkData);
    console.log(`  Chunk ${i} -> ${nodeUrl} (${chunkData.length} bytes)`);

    chunks.push({ index: i, chunkId, nodeUrl, size: chunkData.length, hash });
  }

  // Register file metadata with the coordinator
  await httpRequest(`${COORDINATOR_URL}/files`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ filename, totalSize, chunks }),
  });

  console.log(`Upload complete: ${filename}`);
}

What does the upload function do?

  • Reads the entire file into memory, then splits it into 1 MB pieces using Buffer.slice().
  • Computes a SHA-256 hash of each chunk so the downloader can verify integrity later.
  • Asks the coordinator for storage node assignments via POST /allocate. The coordinator's round-robin logic decides which node gets each chunk.
  • Sends each chunk to its assigned node via PUT /chunks/:id, then registers the complete file metadata with the coordinator via POST /files.
  • Save client.js.

You won't see a change yet. The next section adds the download and list commands that complete the CLI.

Seeing a syntax error when you save?

  • Check that the upload function's closing brace } is on its own line at the end.
  • Make sure you have backticks (`) around template literal strings, not regular quotes.

Why is my client.js showing a syntax error?

Implement the download command

Downloading reverses the upload process. The client fetches file metadata from the coordinator, downloads each chunk from its storage node, verifies integrity using the stored hash, and reassembles the original file.

  • Below the upload function in client.js, add the downloadChunk helper function:
function downloadChunk(nodeUrl, chunkId) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(`${nodeUrl}/chunks/${chunkId}`);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname,
      method: 'GET',
    };

    const req = http.request(reqOptions, (res) => {
      if (res.statusCode !== 200) {
        reject(new Error(`HTTP ${res.statusCode}`));
        res.resume();
        return;
      }
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => resolve(Buffer.concat(chunks)));
    });

    req.on('error', reject);
    req.end();
  });
}

What does downloadChunk do?

downloadChunk sends a GET request to a storage node for a specific chunk ID. If the response is not 200, it rejects with an error. Otherwise it collects the raw binary data and resolves with the complete Buffer.

  • Below downloadChunk, add the full download command function:
async function download(filename, outputPath) {
  // Fetch file metadata from the coordinator
  const metaRes = await httpRequest(
    `${COORDINATOR_URL}/files/${encodeURIComponent(filename)}`
  );

  if (metaRes.statusCode === 404) {
    console.error(`File not found: ${filename}`);
    process.exit(1);
  }

  const fileMeta = JSON.parse(metaRes.body.toString());
  console.log(
    `Downloading: ${filename} (${fileMeta.totalSize} bytes, ${fileMeta.chunks.length} chunks)`
  );

  // Download each chunk and verify its integrity
  const chunkBuffers = [];
  for (const chunk of fileMeta.chunks) {
    const data = await downloadChunk(chunk.nodeUrl, chunk.chunkId);
    const hash = crypto.createHash('sha256').update(data).digest('hex');

    if (hash !== chunk.hash) {
      console.error(`Integrity check failed for chunk ${chunk.index}!`);
      process.exit(1);
    }

    console.log(`  Chunk ${chunk.index} <- ${chunk.nodeUrl} (verified)`);
    chunkBuffers.push(data);
  }

  // Reassemble the file from chunks
  const fileData = Buffer.concat(chunkBuffers);
  const outputFile = outputPath || filename;
  fs.writeFileSync(outputFile, fileData);
  console.log(`Download complete: ${outputFile}`);
}

How does integrity verification work?

After downloading each chunk, the client computes a fresh SHA-256 hash and compares it to the hash stored during upload. If the hashes do not match, the data was corrupted in transit or on disk, and the download stops immediately.

This is the same pattern production systems like S3 use to guarantee data integrity across distributed storage.

  • Save client.js.

You won't see a change yet. The next chunk adds the list command and the CLI routing that makes everything runnable.

Add the list command and verify the CLI

The final piece is the list command (to see all stored files) and the command routing switch that parses arguments and dispatches to the correct function.

  • Below the download function, add the list function and the command routing logic:
async function list() {
  const res = await httpRequest(`${COORDINATOR_URL}/files`);
  const { files } = JSON.parse(res.body.toString());

  if (files.length === 0) {
    console.log('No files stored.');
    return;
  }

  console.log('Stored files:');
  for (const file of files) {
    console.log(
      `  ${file.filename} (${file.totalSize} bytes, ${file.chunkCount} chunks, uploaded ${file.uploadedAt})`
    );
  }
}

// Parse command-line arguments and dispatch
const command = process.argv[2];
const args = process.argv.slice(3);

switch (command) {
  case 'upload':
    if (!args[0]) {
      console.error('Usage: node client.js upload <filepath>');
      process.exit(1);
    }
    upload(args[0]).catch(console.error);
    break;
  case 'download':
    if (!args[0]) {
      console.error('Usage: node client.js download <filename> [output-path]');
      process.exit(1);
    }
    download(args[0], args[1]).catch(console.error);
    break;
  case 'list':
    list().catch(console.error);
    break;
  default:
    console.log('Distributed File Storage Client');
    console.log('');
    console.log('Usage:');
    console.log('  node client.js upload <filepath>');
    console.log('  node client.js download <filename> [output-path]');
    console.log('  node client.js list');
}

What does the command routing do?

  • process.argv[2] grabs the first argument after node client.js (the command name).
  • The switch dispatches to upload, download, or list. If no command is given, it prints usage instructions.
  • Each command validates its arguments before running. Missing a required argument prints a helpful error message.
  • Save client.js.
  • Verify the CLI works by running this command in your terminal (make sure you are inside the distributed-storage folder):
node client.js

You should see usage instructions listing the three available commands: upload, download, and list.

Not seeing the usage instructions?

  • Make sure your terminal is inside the distributed-storage folder. Run ls and confirm you see client.js in the output.
  • If you see a syntax error, check that all template literals use backticks and all parentheses and braces are balanced.
  • If you see Cannot find module, double check the file is named exactly client.js (not Client.js or client.ts).

Why is my client.js not running?

✔️ Awesome, I've got everything!

Great. Double check you have saved client.js and the usage output looks correct.

ⓧ I'd like to double check the full code

Here is the complete client.js file for comparison:

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');
const crypto = require('node:crypto');

const COORDINATOR_URL = 'http://localhost:4000';
const CHUNK_SIZE = 1024 * 1024;

function httpRequest(url, options = {}) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(url);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname + parsedUrl.search,
      method: options.method || 'GET',
      headers: options.headers || {},
    };

    const req = http.request(reqOptions, (res) => {
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => {
        const body = Buffer.concat(chunks);
        resolve({ statusCode: res.statusCode, body });
      });
    });

    req.on('error', reject);

    if (options.body) {
      req.write(options.body);
    }

    req.end();
  });
}

function sendChunk(nodeUrl, chunkId, chunkData) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(`${nodeUrl}/chunks/${chunkId}`);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname,
      method: 'PUT',
      headers: { 'Content-Length': chunkData.length },
    };

    const req = http.request(reqOptions, (res) => {
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => {
        const body = Buffer.concat(chunks).toString();
        resolve(JSON.parse(body));
      });
    });

    req.on('error', reject);
    req.write(chunkData);
    req.end();
  });
}

function downloadChunk(nodeUrl, chunkId) {
  return new Promise((resolve, reject) => {
    const parsedUrl = new URL(`${nodeUrl}/chunks/${chunkId}`);
    const reqOptions = {
      hostname: parsedUrl.hostname,
      port: parsedUrl.port,
      path: parsedUrl.pathname,
      method: 'GET',
    };

    const req = http.request(reqOptions, (res) => {
      if (res.statusCode !== 200) {
        reject(new Error(`HTTP ${res.statusCode}`));
        res.resume();
        return;
      }
      const chunks = [];
      res.on('data', (chunk) => chunks.push(chunk));
      res.on('end', () => resolve(Buffer.concat(chunks)));
    });

    req.on('error', reject);
    req.end();
  });
}

async function upload(filePath) {
  const fullPath = path.resolve(filePath);

  if (!fs.existsSync(fullPath)) {
    console.error(`File not found: ${fullPath}`);
    process.exit(1);
  }

  const fileData = fs.readFileSync(fullPath);
  const filename = path.basename(fullPath);
  const totalSize = fileData.length;

  console.log(`Uploading: ${filename} (${totalSize} bytes)`);

  const chunkDataList = [];
  for (let i = 0; i < fileData.length; i += CHUNK_SIZE) {
    chunkDataList.push(fileData.slice(i, i + CHUNK_SIZE));
  }

  console.log(`Split into ${chunkDataList.length} chunk(s)`);

  const allocRes = await httpRequest(`${COORDINATOR_URL}/allocate`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ chunkCount: chunkDataList.length }),
  });

  const { allocations } = JSON.parse(allocRes.body.toString());

  const chunks = [];
  for (let i = 0; i < chunkDataList.length; i++) {
    const chunkData = chunkDataList[i];
    const hash = crypto.createHash('sha256').update(chunkData).digest('hex');
    const chunkId = `${filename}-chunk-${i}-${hash.slice(0, 8)}`;
    const nodeUrl = allocations[i];

    await sendChunk(nodeUrl, chunkId, chunkData);
    console.log(`  Chunk ${i} -> ${nodeUrl} (${chunkData.length} bytes)`);

    chunks.push({ index: i, chunkId, nodeUrl, size: chunkData.length, hash });
  }

  await httpRequest(`${COORDINATOR_URL}/files`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ filename, totalSize, chunks }),
  });

  console.log(`Upload complete: ${filename}`);
}

async function download(filename, outputPath) {
  const metaRes = await httpRequest(
    `${COORDINATOR_URL}/files/${encodeURIComponent(filename)}`
  );

  if (metaRes.statusCode === 404) {
    console.error(`File not found: ${filename}`);
    process.exit(1);
  }

  const fileMeta = JSON.parse(metaRes.body.toString());
  console.log(
    `Downloading: ${filename} (${fileMeta.totalSize} bytes, ${fileMeta.chunks.length} chunks)`
  );

  const chunkBuffers = [];
  for (const chunk of fileMeta.chunks) {
    const data = await downloadChunk(chunk.nodeUrl, chunk.chunkId);
    const hash = crypto.createHash('sha256').update(data).digest('hex');

    if (hash !== chunk.hash) {
      console.error(`Integrity check failed for chunk ${chunk.index}!`);
      process.exit(1);
    }

    console.log(`  Chunk ${chunk.index} <- ${chunk.nodeUrl} (verified)`);
    chunkBuffers.push(data);
  }

  const fileData = Buffer.concat(chunkBuffers);
  const outputFile = outputPath || filename;
  fs.writeFileSync(outputFile, fileData);
  console.log(`Download complete: ${outputFile}`);
}

async function list() {
  const res = await httpRequest(`${COORDINATOR_URL}/files`);
  const { files } = JSON.parse(res.body.toString());

  if (files.length === 0) {
    console.log('No files stored.');
    return;
  }

  console.log('Stored files:');
  for (const file of files) {
    console.log(
      `  ${file.filename} (${file.totalSize} bytes, ${file.chunkCount} chunks, uploaded ${file.uploadedAt})`
    );
  }
}

const command = process.argv[2];
const args = process.argv.slice(3);

switch (command) {
  case 'upload':
    if (!args[0]) {
      console.error('Usage: node client.js upload <filepath>');
      process.exit(1);
    }
    upload(args[0]).catch(console.error);
    break;
  case 'download':
    if (!args[0]) {
      console.error('Usage: node client.js download <filename> [output-path]');
      process.exit(1);
    }
    download(args[0], args[1]).catch(console.error);
    break;
  case 'list':
    list().catch(console.error);
    break;
  default:
    console.log('Distributed File Storage Client');
    console.log('');
    console.log('Usage:');
    console.log('  node client.js upload <filepath>');
    console.log('  node client.js download <filename> [output-path]');
    console.log('  node client.js list');
}

Your client CLI is complete. Next up, you will start the full system and watch files get split, distributed, and reassembled in real time.

Run the Full System

Your client can now split files into chunks, distribute them across storage nodes, and reassemble them on download. All three services are written and ready to talk to each other over HTTP.

In this step, you will launch the full distributed system and prove it works end-to-end. You will start multiple services in separate terminals, upload a file, watch it get distributed, and download it back with integrity verification.

In this step, get ready to:
  • Start two storage nodes and a coordinator in separate terminals.
  • Upload a test file and observe chunk distribution across nodes.
  • Download the file and verify byte-for-byte integrity.
Start the distributed system in separate terminals

A distributed system means multiple independent processes running simultaneously. You need three separate terminal windows to simulate three independent services communicating over the network.

  • Open three separate terminal windows. You can do this by pressing Cmd+Space (macOS) or the Windows key (Windows), typing your terminal app name, and pressing Enter three times.
  • In each terminal, navigate to your distributed-storage folder by running:
cd ~/Desktop/distributed-storage
  • In the first terminal, start storage node 1 by running:
node storage-node.js 4001 ./storage-data-1
  • You should see output confirming the node is running on port 4001 and storing chunks in storage-data-1.
  • In the second terminal, start storage node 2 by running:
node storage-node.js 4002 ./storage-data-2
  • You should see the same confirmation, this time on port 4002.
  • In the third terminal, start the coordinator and register both storage nodes by running:
node coordinator.js 4000 http://localhost:4001 http://localhost:4002
  • You should see output confirming the coordinator is running on port 4000 with both storage node URLs listed.

What is happening here?

You now have three independent processes running, each on its own port. The coordinator on port 4000 knows about both storage nodes. When a client asks for storage allocations, the coordinator distributes work between them using round-robin.

This is the same architecture pattern used by Google File System and HDFS. The coordinator is the "NameNode" that tracks where data lives. The storage nodes are the "DataNodes" that hold the actual bytes.

Seeing "address already in use"?

This means another process is already running on that port. Close any previous instances of the storage nodes or coordinator that might still be running from earlier testing.

Press Ctrl+C in any terminal where an old process is running to stop it, then try starting the service again.

Help me debug why my port is already in use.

Upload a test file and observe the distribution

With all three services running, you can now use the client to upload a file. The client will split it into chunks, ask the coordinator where to put them, and send each chunk to the assigned storage node.

  • In the third terminal (the one running the coordinator), press Ctrl+C to stop it temporarily. You will use this terminal for client commands.

Keep the coordinator running

Actually, you need the coordinator running for uploads to work. Open a fourth terminal window instead, navigate to your distributed-storage folder, and use that for client commands. Keep all three services running in their respective terminals.

  • Open a fourth terminal window and navigate to your project folder by running:
cd ~/Desktop/distributed-storage
  • Upload your sample file by running:
node client.js upload ./test-files/sample.txt
  • You should see output showing the file being split into chunks and each chunk being sent to a storage node URL.
  • Verify chunks are distributed across both storage nodes by running:
ls ./storage-data-1
ls ./storage-data-2
  • You should see chunk files in at least one of these directories. If your sample file is small (under 1MB), all chunks may land on one node. If it is larger, chunks will be split across both.
  • Check the coordinator's file registry by running:
node client.js list
  • You should see sample.txt listed with its size, chunk count, and upload timestamp.

Why round-robin distribution?

The coordinator assigns chunks to storage nodes in a rotating pattern: chunk 0 goes to node 1, chunk 1 goes to node 2, chunk 2 goes back to node 1, and so on. This ensures an even spread of data across all available nodes.

Production systems like HDFS use more sophisticated strategies (like consistent hashing) to handle node additions and removals gracefully. Round-robin is the simplest distribution strategy that still demonstrates the core pattern.

Seeing "connect ECONNREFUSED"?

This means the client cannot reach the coordinator or a storage node. Make sure all three services (two storage nodes and the coordinator) are still running in their respective terminals.

Check that you started the coordinator with the correct storage node URLs: http://localhost:4001 and http://localhost:4002.

Help me debug connection refused errors in my distributed system.

Download the file and verify integrity

The real test of a distributed storage system is whether you can get your data back intact. The download process fetches each chunk from its storage node, verifies the SHA-256 hash, and reassembles the original file.

  • Download the file to a new location by running:
node client.js download sample.txt ./test-files/downloaded.txt
  • You should see output showing each chunk being downloaded and verified. The word "verified" next to each chunk confirms the SHA-256 hash matched.
  • Confirm the downloaded file is identical to the original by comparing them. On Windows, run:
fc ./test-files/sample.txt ./test-files/downloaded.txt
  • If the files are identical, you will see a message confirming no differences were found.

What does SHA-256 verification prove?

When the client uploaded each chunk, it computed a SHA-256 hash of the raw bytes and stored it in the coordinator's metadata. During download, the client recomputes the hash of the received data and compares it to the stored hash.

If even a single byte was corrupted during storage or transfer, the hash would not match and the download would fail. This is the same integrity mechanism used by Git, Docker image layers, and cloud storage services.

Download failing with "File not found"?

Make sure you are using the exact filename that was registered during upload. The download command uses the base filename (e.g. sample.txt), not the full path.

Run node client.js list to check the exact filename stored in the coordinator's registry. The coordinator stores metadata in memory, so if you restarted it between upload and download, the registry is empty and you need to upload again.

Help me figure out why my download is failing.

Secret mission

Add Chunk Replication

Your distributed storage system works, but what happens when a storage node goes down? Right now, each chunk lives on exactly one node. If that node crashes, the data is gone. In this secret mission, you will add replication so each chunk is stored on two nodes, with automatic failover on download.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Running Node.js processes (coordinator on port 4000, storage nodes on ports 4001 and 4002).
  • The distributed-storage project folder on your Desktop.
  • Node.js v24 LTS installation.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with your distributed storage system.

  • Your storage nodes and coordinator will keep running until you close their terminal windows or press Ctrl+C.
  • All uploaded chunks remain in storage-data-1 and storage-data-2.
  • Node.js stays installed and ready for future projects.

Pause - I'll come back to this later

Shut down the running processes to free up terminal windows and ports. Your project files stay on disk so you can restart everything later.

  • Switch to each terminal window running a storage node or the coordinator and press Ctrl+C to stop the process.
  • Repeat for all three terminals (two storage nodes and one coordinator).
  • To restart later, open three terminals, navigate to your distributed-storage folder, and run the same startup commands from Step 5.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes your project files and optionally removes Node.js from your system.

  • Stop all running processes by pressing Ctrl+C in each terminal window (coordinator and both storage nodes).
  • Delete the project folder on macOS/Linux by running:
cd ~/Desktop
rm -rf distributed-storage
  • Delete the project folder on Windows by running:
cd ~\Desktop
rmdir /s /q distributed-storage
  • To uninstall Node.js on Windows, open **Settings**, select **Apps**, then **Installed apps**, search for Node.js, and click **Uninstall**.

Nice Work!

Nice Work!

You just built a distributed file storage system from scratch using nothing but Node.js built-in modules. Your system splits files into chunks, distributes them across independent storage nodes, and reassembles them with cryptographic integrity verification.

You've learned how to:

  • Build a multi-service distributed system with three independent services (coordinator, storage nodes, client) communicating over HTTP.
  • Implement the metadata/data plane separation pattern used by production systems like Google File System, HDFS, and Amazon S3.
  • Use Node.js streams for efficient I/O and SHA-256 integrity verification to detect data corruption across a distributed system.
  • Secret Mission: Added chunk replication with automatic failover so your system stays available even when a storage node goes down.

Ready to quiz yourself?