Build a File Stream Processor

Stream file uploads and reads with Node.js Transform streams and pipeline.

Introduction

30 Second Summary

Think about streaming a video versus downloading the whole thing first. That difference between getting data piece by piece and waiting for everything to arrive is exactly what separates streaming servers from traditional buffered ones.

In this project, you will build a Node.js HTTP server that processes text files using streams. You will experience firsthand the difference between a server that goes silent while loading a file and one that delivers annotated results to your browser line by line.

What You'll Build

You will upload a text file in your browser, watch live progress updates tick up byte by byte, then request the file back and see annotated lines appear one by one instead of waiting for the entire file to load.

By the end of this project, you'll have:

  • A file upload that streams live progress back to the browser in real time, showing bytes received as they flow in.
  • A read endpoint that delivers annotated lines (line numbers, word counts) progressively through a custom Transform stream, so the first line appears almost instantly.
  • A single pipeline() call that wires your file reader, transform, and HTTP response together with automatic error handling and backpressure.
  • Secret Mission: Add a streaming search endpoint that highlights matching lines as they flow through a composed pipeline of multiple Transform streams.

Are there any prerequisites?

This project assumes familiarity with Node.js basics (modules, callbacks, HTTP servers) and REST architecture. You do not need any external packages. The entire project uses only built-in Node.js modules.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

This step is your moment to commit to the goal. You already know how REST works: send a request, wait, get a full response back. This project challenges that pattern by making data flow continuously instead of arriving all at once.

Set Up the Project

Every streaming project starts with the same foundation: a project folder, a confirmed runtime, and a minimal server you can hit in the browser.

In this step, you will create the project structure, verify Node.js is ready, and stand up a bare-bones HTTP server that serves a placeholder page.

In this step, get ready to:
  • Create the project folder and confirm Node.js 24.x is installed.
  • Write a minimal HTTP server that serves an HTML page on port 3000.
  • Open the page in your browser and verify it loads.
Create the project folder
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Open a terminal inside VS Code by pressing Ctrl+` (macOS and Windows).
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir file-stream-processor
cd file-stream-processor
  • Confirm Node.js is installed by running:
node --version

You should see v24.x.x printed (any 24.x version is fine).

Don't see a version number?

If your terminal prints command not found or a version lower than 24, revisit the Node.js installation instructions at nodejs.org and install the latest LTS release before continuing.

help me install Node.js 24.x on my machine

  • Create the starter files and the uploads folder by running:
touch server.js index.html
mkdir uploads
  • Confirm the files exist by running:
ls

You should see index.html, server.js, and uploads in the output.

  • Open this folder in VS Code by clicking File in the top menu bar.
  • Click Open Folder.
  • Select the file-stream-processor folder on your Desktop, then click Open.

Why open the folder this way?

Opening the folder (instead of individual files) lets VS Code's Explorer sidebar show your full project structure. You will see server.js, index.html, and the uploads folder listed in the left panel.

Write the minimal HTTP server

Your server needs to do one thing for now: serve index.html when someone visits the root URL.

  • Open index.html in the Explorer sidebar and paste the following placeholder page:
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>File Stream Processor</title>
</head>
<body>
  <h1>File Stream Processor</h1>
  <p>Coming soon...</p>
</body>
</html>

What does this page do?

This is a simple placeholder. You will replace it with a full upload form and streaming output display in later steps.

  • Save index.html by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Open server.js in the Explorer sidebar and paste the following code:
const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const PORT = 3000;

const server = http.createServer((req, res) => {
  // Serve index.html for GET requests to the root URL
  if (req.method === 'GET' && req.url === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }

  // Return 404 for any other route
  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

What does this code do?

  • node:http, node:fs, and node:path are built-in Node.js modules. No installs needed.
  • fs.createReadStream() opens the HTML file as a Readable stream instead of loading it entirely into memory.
  • .pipe(res) connects the file stream directly to the HTTP response (a Writable stream). Data flows from disk to client without buffering the whole file.
  • Any URL other than / gets a 404 response.
  • Save server.js.

Seeing red squiggles in VS Code?

If VS Code shows warnings about require or __dirname, that is a TypeScript type-checking hint. It does not affect the code running. You can safely ignore it.

help me understand why VS Code shows warnings about require in my Node.js file

Before you run the server, what do you think will happen when you visit http://localhost:3000 in a browser?

  • Start the server by running this command in your terminal:
node server.js

You should see this output in your terminal:

Server running at http://localhost:3000
  • Open your browser and navigate to http://localhost:3000.

You should see a page with the heading "File Stream Processor" and the text "Coming soon..." displayed.

Page not loading in the browser?

Make sure you typed http://localhost:3000 exactly (not https). Check your terminal to confirm the server printed the "Server running" message. If the terminal shows an error about port 3000 already in use, stop whatever is using that port and try again.

help me debug why localhost:3000 is not loading my page

✔️ Awesome, I've got everything!

Great. Make sure you have saved both files.

ⓧ I'd like to double check the full code

Here is what each file should contain at this point.

server.js:

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const PORT = 3000;

const server = http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }

  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

index.html:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>File Stream Processor</title>
</head>
<body>
  <h1>File Stream Processor</h1>
  <p>Coming soon...</p>
</body>
</html>

Your project skeleton is ready and the server is confirmed working. Next up, you will build a file upload endpoint and experience the buffered approach firsthand.

Build the Buffered Upload and Read

Your server is alive and serving a placeholder page. Now you need it to actually do something with files.

In this step, you will build the traditional REST pattern: the client uploads a file, the server collects every byte into memory, writes it to disk, and on a separate request reads the whole thing back. This "buffer everything first" approach works. Later steps will reveal why it falls short.

In this step, get ready to:
  • Add a file upload form and output panel to index.html.
  • Create a POST /upload endpoint that buffers the file and writes it to disk.
  • Create a GET /read endpoint that reads the file and sends it back in one shot.
Add the upload form and output panel to index.html

The current index.html is just a placeholder. You need a file input, buttons, and an output area so the browser can communicate with your server endpoints.

  • In VS Code, open index.html and select all the existing content.
  • Replace it with the following HTML structure:
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>File Stream Processor</title>
  <style>
    body { font-family: monospace; max-width: 800px; margin: 40px auto; padding: 0 20px; }
    .section { margin: 20px 0; padding: 15px; border: 1px solid #ddd; border-radius: 4px; }
    button { padding: 8px 16px; margin: 5px; cursor: pointer; }
    #output { background: #1e1e1e; color: #0f0; padding: 15px; min-height: 200px; white-space: pre-wrap; border-radius: 4px; }
  </style>
</head>
<body>
  <h1>File Stream Processor</h1>
  <div class="section">
    <h2>1. Upload a Text File</h2>
    <input type="file" id="fileInput" accept=".txt,.csv,.log,.md">
    <button onclick="uploadFile()">Upload</button>
  </div>
  <div class="section">
    <h2>2. Read File</h2>
    <button onclick="readFile()">Read File</button>
  </div>
  <div class="section">
    <h2>Output</h2>
    <div id="output">Waiting for action...</div>
  </div>
  <script>
  </script>
</body>
</html>

What does this HTML do?

  • The file input (id="fileInput") accepts text-based files like .txt, .csv, .log, and .md.
  • The #output div acts as a terminal-style display area where server responses will appear.
  • The empty <script> block is where you will add the JavaScript that talks to your server next.
  • Save index.html.
  • Refresh http://localhost:3000 in your browser. You should see an upload section with a file picker and button, a Read File button, and a dark green-on-black output panel.

Still seeing the old placeholder page?

  • Make sure you saved the file in VS Code (Cmd+S on macOS, Ctrl+S on Windows).
  • Hard-refresh the browser with Cmd+Shift+R (macOS) or Ctrl+Shift+R (Windows) to bypass any cache.
  • help me debug my HTML not loading in the browser

Now add the JavaScript that will call your server endpoints. These functions send the file to the server and display whatever comes back.

  • In index.html, find the empty <script> block and paste the following code between the opening and closing tags:
    const output = document.getElementById('output');
    async function uploadFile() {
      const file = document.getElementById('fileInput').files[0];
      if (!file) { output.textContent = 'Please select a file first.'; return; }
      output.textContent = 'Uploading...';
      const response = await fetch('/upload', { method: 'POST', body: file });
      output.textContent = await response.text();
    }
    async function readFile() {
      output.textContent = 'Reading...';
      const response = await fetch('/read');
      output.textContent = await response.text();
    }

What does this JavaScript do?

  • uploadFile() sends the raw file to POST /upload using the Fetch API. The file goes as the request body with no multipart encoding.
  • readFile() calls GET /read and dumps the full response text into the output panel.
  • Both functions display the server's plain-text response directly in the #output div.
  • Save index.html.

The buttons will not work yet because the server does not have the /upload or /read endpoints. You will add those next.

Add the POST /upload endpoint to server.js

When a file arrives via POST, the req object is a Readable stream. Data arrives in chunks. The buffered approach collects every chunk into an array, concatenates them into a single Buffer, then writes the whole thing to disk.

  • In server.js, find the line const PORT = 3000; and add the following code below it:
const UPLOAD_DIR = path.join(__dirname, 'uploads');

if (!fs.existsSync(UPLOAD_DIR)) {
  fs.mkdirSync(UPLOAD_DIR);
}

function getUploadedFilePath() {
  return path.join(UPLOAD_DIR, 'uploaded.txt');
}

What does this code do?

  • UPLOAD_DIR points to the uploads/ folder you created earlier. The existsSync check ensures it exists even if someone deletes it.
  • getUploadedFilePath() is a helper that returns the path where the uploaded file will be saved. Every upload overwrites the same file for simplicity.
  • Now find the 404 fallback at the bottom of the createServer callback (the line res.writeHead(404, { 'Content-Type': 'text/plain' });). Add the following POST handler above that fallback:
  if (req.method === 'POST' && req.url === '/upload') {
    const filePath = getUploadedFilePath();
    const chunks = [];
    // Collect each incoming data chunk into an array
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      // Combine all chunks into one Buffer and write to disk
      const buffer = Buffer.concat(chunks);
      fs.writeFileSync(filePath, buffer);
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end(`Upload complete! ${buffer.length} bytes received.`);
    });
    req.on('error', (err) => {
      res.writeHead(500);
      res.end(`Upload error: ${err.message}`);
    });
    return;
  }

What does this endpoint do?

  • req.on('data') fires every time a chunk of the file body arrives. Each chunk is pushed into the chunks array.
  • req.on('end') fires when the entire body has been received. Only then does the server assemble the full Buffer and write it.
  • This is the buffered pattern: the server must hold the entire file in memory before it can do anything with it.
  • Save server.js.
  • Switch to your terminal and stop the running server by pressing Ctrl+C.
  • Restart the server by running:
node server.js
  • Refresh http://localhost:3000 in your browser.
  • Click the file picker, select any small .txt file from your computer, and click Upload.
  • You should see a message like Upload complete! 342 bytes received. appear in the output panel (the exact number depends on your file size).

Seeing a network error or no response?

  • Confirm you stopped the old server (Ctrl+C) and restarted it. The server must be restarted after code changes.
  • Check for typos in req.url === '/upload'. The path must match exactly what the JavaScript fetch('/upload', ...) sends.
  • help me debug my upload endpoint not responding
Add the GET /read endpoint

Now you need a way to read the file back. This endpoint loads the entire file into a string using fs.readFileSync() and sends it as one response.

  • In server.js, find the POST /upload handler you just added. Add the following code below its closing brace and above the 404 fallback:
  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    // Read the ENTIRE file into memory as one string
    const content = fs.readFileSync(filePath, 'utf8');
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(content);
    return;
  }

What does this endpoint do?

  • fs.readFileSync() loads the entire file into a single string. Nothing is sent to the client until the whole file is in memory.
  • res.end(content) sends the full string as one HTTP response. The browser receives everything at once.
  • For small files this is fine. For large files, the client stares at a blank screen until the entire file is loaded and sent.
  • Save server.js.
  • Stop the server with Ctrl+C and restart it:
node server.js
  • Refresh the browser page and click Read File.
  • You should see the text content of the file you uploaded earlier appear in the output panel.

Seeing 'No file uploaded yet' when you click Read File?

  • This means uploads/uploaded.txt does not exist. Upload a file first using the Upload button, then try Read File again.
  • help me figure out why my read endpoint cannot find the uploaded file
Test the full upload-then-read flow

You need a sample text file with enough content to make the buffering behaviour visible in later steps. Create one now and run through the complete flow.

  • Open any text editor (or VS Code) and create a new file called sample.txt anywhere on your computer (your Desktop is fine).
  • Paste or type 20 to 30 lines of text. Any content works. Here is an example you can use:
The quick brown fox jumped over the lazy dog.
Node.js streams allow you to process data piece by piece.
This is line three of our sample text file.
Streams are especially useful for large files.
Buffering means holding everything in memory at once.
A Readable stream emits data in chunks.
A Writable stream consumes data in chunks.
A Transform stream modifies data as it passes through.
The pipeline function wires streams together safely.
Backpressure prevents fast producers from overwhelming slow consumers.
This line exists to add more content to our test file.
Each line will eventually get a line number and word count.
Streaming means the first byte arrives before the last byte is ready.
HTTP responses can be sent in chunks using Transfer-Encoding.
Node.js uses an event loop to handle asynchronous I/O.
The fs module provides both sync and async file operations.
readFileSync blocks the event loop until the file is fully read.
createReadStream reads a file without loading it all into memory.
This is line nineteen of our sample file.
We are almost at the end of our sample data.
Line twenty-one adds a bit more variety to the text.
Different line lengths help test the word count feature.
Short line.
This line is intentionally longer to provide contrast with the shorter lines above.
Final line of the sample text file.
  • Save the file.
  • Return to http://localhost:3000 in your browser.
  • Click the file input and select your sample.txt file.
  • Click Upload. You should see a message confirming the bytes received.

Before you click Read File, take a moment. What do you expect to happen? Will the lines appear one by one, or all at once?

  • Click Read File.

Every line of your sample file appears in the output panel instantly, all at once. The server loaded the entire file into memory, then sent it as a single response.

Why did it all appear at once?

readFileSync() blocks the server until the entire file is in memory. Only then does res.end() send everything as one payload. The browser has nothing to display until that moment.

With a 25-line text file, the wait is imperceptible. In the next step, you will simulate what happens when the file is large or processing is slow. That is where the buffered pattern breaks down.

✔️ Awesome, I've got everything!

Great. Make sure both server.js and index.html are saved.

ⓧ I'd like to double check the full code

Here is the complete server.js at this point:

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const UPLOAD_DIR = path.join(__dirname, 'uploads');
const PORT = 3000;

if (!fs.existsSync(UPLOAD_DIR)) {
  fs.mkdirSync(UPLOAD_DIR);
}

function getUploadedFilePath() {
  return path.join(UPLOAD_DIR, 'uploaded.txt');
}

And the createServer callback and listener:

const server = http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }

  if (req.method === 'POST' && req.url === '/upload') {
    const filePath = getUploadedFilePath();
    const chunks = [];
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      const buffer = Buffer.concat(chunks);
      fs.writeFileSync(filePath, buffer);
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end(`Upload complete! ${buffer.length} bytes received.`);
    });
    req.on('error', (err) => {
      res.writeHead(500);
      res.end(`Upload error: ${err.message}`);
    });
    return;
  }

  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    const content = fs.readFileSync(filePath, 'utf8');
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(content);
    return;
  }
  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

And here is the complete index.html:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>File Stream Processor</title>
  <style>
    body { font-family: monospace; max-width: 800px; margin: 40px auto; padding: 0 20px; }
    .section { margin: 20px 0; padding: 15px; border: 1px solid #ddd; border-radius: 4px; }
    button { padding: 8px 16px; margin: 5px; cursor: pointer; }
    #output { background: #1e1e1e; color: #0f0; padding: 15px; min-height: 200px; white-space: pre-wrap; border-radius: 4px; }
  </style>
</head>
<body>
  <h1>File Stream Processor</h1>
  <div class="section">
    <h2>1. Upload a Text File</h2>
    <input type="file" id="fileInput" accept=".txt,.csv,.log,.md">
    <button onclick="uploadFile()">Upload</button>
  </div>
  <div class="section">
    <h2>2. Read File</h2>
    <button onclick="readFile()">Read File</button>
  </div>
  <div class="section">
    <h2>Output</h2>
    <div id="output">Waiting for action...</div>
  </div>
  <script>
    const output = document.getElementById('output');
    async function uploadFile() {
      const file = document.getElementById('fileInput').files[0];
      if (!file) { output.textContent = 'Please select a file first.'; return; }
      output.textContent = 'Uploading...';
      const response = await fetch('/upload', { method: 'POST', body: file });
      output.textContent = await response.text();
    }
    async function readFile() {
      output.textContent = 'Reading...';
      const response = await fetch('/read');
      output.textContent = await response.text();
    }
  </script>
</body>
</html>

Your server can accept a file upload and read it back. The entire file loads into memory before anything is sent to the browser. For 25 lines of text, that feels instant. In the next step, you will add a processing delay that exposes exactly how long the browser waits with no feedback.

Feel the Buffering Problem

Your upload-then-read flow works, but the file you tested was tiny. With a small file, the server responds instantly and everything feels fine. The buffering limitation is invisible.

In the real world, files are large and processing takes time. In this step, you will add a deliberate per-line delay to simulate slow I/O. This forces the buffering problem to reveal itself: the browser goes silent for several seconds, then dumps all the content at once.

In this step, get ready to:
  • Add a simulated processing delay to the GET /read endpoint.
  • Observe the buffering problem in the browser and Network tab.
Add a sleep helper function

To simulate slow processing, you need a way to pause execution for a set number of milliseconds. JavaScript does not have a built-in sleep function, but you can create one with a Promise and setTimeout.

  • In server.js, find the getUploadedFilePath() function.
  • Add this sleep helper directly below it:
function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

What does this code do?

This creates a Promise that resolves after the given number of milliseconds. Combined with await, it pauses execution inside an async function. You will use it to simulate slow line-by-line processing.

Make the server callback async

To use await sleep() inside your request handler, the callback must be an async function.

  • In server.js, scroll down to the http.createServer line.
  • Find this line:
const server = http.createServer((req, res) => {
  • Add async before (req, res), so the line now looks like this:
const server = http.createServer(async (req, res) => {
Replace the read handler with a delayed version

Now you will change the /read handler so it splits the file into lines, processes each one with a 200ms delay, buffers all results, and only sends the response once every line is finished.

  • Inside the if (req.method === 'GET' && req.url === '/read') block, find these three lines after the 404 check:
    const content = fs.readFileSync(filePath, 'utf8');
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(content);
  • Replace those three lines with the following code:
    // Read entire file into memory
    const content = fs.readFileSync(filePath, 'utf8');
    // Split into lines and process each with a delay
    const lines = content.split('\n');
    const processed = [];
    for (const line of lines) {
      await sleep(200);
      processed.push(line);
    }
    // Only NOW send everything back at once
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(processed.join('\n'));

What does this code do?

This simulates what happens when a server processes a large file or talks to a slow database. Each line takes 200ms to "process." If your sample file has 20 lines, that is 4 full seconds of silence before the browser receives a single byte.

  • content.split('\n') breaks the file into individual lines.
  • await sleep(200) pauses 200ms per line to simulate slow I/O.
  • res.end(processed.join('\n')) sends ALL processed lines only after the loop finishes. Nothing reaches the browser until every line is done.
  • Save server.js.

Seeing a syntax error when you save?

  • Make sure you added async before (req, res) in the createServer line. Without async, await is not valid syntax.
  • Check that your sleep function is defined outside the createServer callback, not inside it.

Help me fix a syntax error in my server.js after adding async and await sleep

✔️ Awesome, I've got everything!

Great. Double check that you have saved server.js.

ⓧ I'd like to double check the full code

Here is the full server.js at this point. Scroll up and confirm each section is present in this order: requires, constants, getUploadedFilePath, sleep, the async createServer callback with the modified /read handler, and server.listen.

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const UPLOAD_DIR = path.join(__dirname, 'uploads');
const PORT = 3000;

if (!fs.existsSync(UPLOAD_DIR)) {
  fs.mkdirSync(UPLOAD_DIR);
}

function getUploadedFilePath() {
  return path.join(UPLOAD_DIR, 'uploaded.txt');
}

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}
const server = http.createServer(async (req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }

  if (req.method === 'POST' && req.url === '/upload') {
    const filePath = getUploadedFilePath();
    const chunks = [];
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      const buffer = Buffer.concat(chunks);
      fs.writeFileSync(filePath, buffer);
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end(`Upload complete! ${buffer.length} bytes received.`);
    });
    req.on('error', (err) => {
      res.writeHead(500);
      res.end(`Upload error: ${err.message}`);
    });
    return;
  }
  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    const content = fs.readFileSync(filePath, 'utf8');
    const lines = content.split('\n');
    const processed = [];
    for (const line of lines) {
      await sleep(200);
      processed.push(line);
    }
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(processed.join('\n'));
    return;
  }

  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});
Observe the buffering problem

Time to see what buffering actually feels like from the user's perspective.

  • Switch back to your terminal in VS Code.
  • Stop the running server by pressing Ctrl+C.
  • Restart the server by running:
node server.js
  • Return to your browser at http://localhost:3000.
  • Open DevTools by pressing Cmd+Option+I (macOS) or Ctrl+Shift+I (Windows).
  • Click the Network tab in DevTools.

Before you click Read File, think about what you expect to see. Your sample file has around 20 lines. Each line takes 200ms to "process." How long will the browser wait before anything appears?

  • Click the Read File button.

Watch the output area. It says "Reading..." and then nothing happens. The browser sits silently for several seconds. Then all the content appears in a single burst.

  • Look at the Network tab. Click the read request to see its timing.

You will see the request spent almost all its time in "Waiting for server response" (the TTFB column). The actual download was nearly instant because everything arrived in one chunk at the end.

Why is this a problem?

The server knew the first line's content within 200ms. It could have sent it immediately. Instead, it held everything hostage until the last line was done.

For a user waiting on file processing, AI token generation, or log output, this pattern creates dead silence followed by a data dump. Streaming fixes this by sending each piece the moment it is ready.

You have now felt the buffering problem firsthand. The server knew the answer to the first line almost immediately, but the user saw nothing until every single line was processed. In the next step, you will fix this by streaming the response so the first byte arrives within milliseconds.

Stream the Response with ReadStream

In the previous step, you felt the buffering problem firsthand. The server loaded the entire file, processed every line with a delay, then dumped everything to the browser in one burst. The browser sat there showing nothing for seconds.

The fix is surprisingly simple. Instead of reading the whole file into memory, you open a ReadStream that reads the file in small chunks and pipes each chunk directly to the HTTP response. The browser starts receiving data the moment the first chunk is read from disk.

In this step, get ready to:
  • Replace the buffered readFileSync with a streaming createReadStream piped to the response.
  • Update the browser JavaScript to display chunks as they arrive using the Fetch API's stream reader.
  • Test that file content now appears progressively with near-instant first byte delivery.
Replace the buffered read with a streaming read

The server currently calls readFileSync to load the entire file into a string, loops through every line with a 200ms delay, and only then sends the response. You will replace all of that with two lines: open a ReadStream, pipe it to res.

  • In server.js, find the sleep helper function:
function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}
  • Delete the entire sleep function. You no longer need it because there is no simulated delay in the streaming approach.
  • Next, find this line near the top of your createServer call:
const server = http.createServer(async (req, res) => {
  • Remove the async keyword so the line looks like this:
const server = http.createServer((req, res) => {
  • Now find the entire GET /read handler block. It starts with:
  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    const content = fs.readFileSync(filePath, 'utf8');
    const lines = content.split('\n');
    const processed = [];
    for (const line of lines) {
      await sleep(200);
      processed.push(line);
    }
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(processed.join('\n'));
    return;
  }
  • Replace that entire block with the streaming version below:
  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    // Stream the file directly to the HTTP response
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    const readStream = fs.createReadStream(filePath);
    readStream.pipe(res);
    return;
  }

What does this code do?

Instead of loading the entire file into a string with readFileSync, you now open a ReadStream that reads the file in 64 KB chunks (the default highWaterMark).

The .pipe(res) call connects the ReadStream's output directly to the HTTP response. Each chunk flows to the browser the moment it is read from disk. Node.js automatically uses chunked transfer encoding because you did not set a Content-Length header.

  • Save server.js.
  • Switch to your terminal and stop the running server by pressing Ctrl+C.
  • Restart the server by running:
node server.js
  • Return to your browser at http://localhost:3000 and click Read File.

The file content appears instantly now. No more multi-second wait. The old browser JavaScript still works because it calls response.text() which collects all chunks before displaying. You will fix that next.

Still seeing a delay or an error?

  • Make sure you removed the async keyword from the createServer callback. Leaving it is harmless but check that you also deleted the await sleep(200) line and the entire old /read handler.
  • If you see Cannot find module, confirm you saved server.js before restarting.
  • If you see No file uploaded yet., re-upload your sample file first.

Help me debug my streaming server

Update the browser to consume chunks progressively

The server is now streaming, but the browser still waits for the entire response before showing anything. That is because response.text() buffers the full body into a single string. You need the browser to read the response body as a stream using the Fetch API's response.body.getReader().

  • In index.html, find the #output CSS rule:
#output { background: #1e1e1e; color: #0f0; padding: 15px; min-height: 200px; white-space: pre-wrap; border-radius: 4px; }
  • Replace it with this updated version that adds a max height, scroll, and word wrapping. Also add the new .status rule directly below it:
#output { background: #1e1e1e; color: #0f0; padding: 15px; min-height: 200px; max-height: 500px; overflow-y: auto; white-space: pre-wrap; word-wrap: break-word; border-radius: 4px; }
.status { color: #666; font-style: italic; }
  • Next, find the second <div class="section"> block that contains the Read File button:
  <div class="section">
    <h2>2. Read File</h2>
    <button onclick="readFile()">Read File</button>
  </div>
  • Replace it with this updated version that references the new streaming function:
  <div class="section">
    <h2>2. Read File (Streamed)</h2>
    <button onclick="readFileStreamed()">Read File</button>
  </div>
  • Find the output <div>:
    <div id="output">Waiting for action...</div>
  • Replace it with a styled status span:
    <div id="output"><span class="status">Waiting for action...</span></div>
  • Now replace the entire <script> block (everything between <script> and </script>) with the streaming version below:
    const output = document.getElementById('output');

    // Helper to clear the output panel
    function clearOutput() { output.textContent = ''; }

    async function uploadFile() {
      const file = document.getElementById('fileInput').files[0];
      if (!file) { output.textContent = 'Please select a file first.'; return; }
      clearOutput();
      output.textContent = 'Uploading...';
      const response = await fetch('/upload', { method: 'POST', body: file });
      output.textContent = await response.text();
    }

    // Read the file using the Fetch streaming API
    async function readFileStreamed() {
      clearOutput();
      output.textContent = 'Reading file (streamed)...\n\n';
      const response = await fetch('/read');
      if (!response.ok) { output.textContent += await response.text(); return; }
      // Get a reader from the response body stream
      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        // Append each decoded chunk to the output as it arrives
        output.textContent += decoder.decode(value, { stream: true });
      }
      output.textContent += '\n--- Stream complete ---';
    }

What does this code do?

  • response.body.getReader() returns a ReadableStreamDefaultReader that lets you pull chunks from the HTTP response one at a time.
  • Each call to reader.read() returns a { done, value } pair. When done is true, the stream has ended.
  • TextDecoder with { stream: true } handles multi-byte characters that might be split across chunks.
  • Each chunk is appended to output.textContent immediately, so the learner sees text appear as it flows in rather than waiting for the full response.
  • Save index.html.

Seeing a blank page or console error?

  • Check that the onclick attribute on the Read File button says readFileStreamed() (not the old readFile()). A mismatched name causes a ReferenceError in the console.
  • Confirm the <script> tags still wrap your JavaScript. If you accidentally deleted one, the page breaks silently.

Help me fix my browser streaming code

Test the streaming flow

Both the server and the browser are now wired for streaming. Before you test, think about what you expect to see.

  • Refresh your browser at http://localhost:3000.
  • Before clicking Read File: do you think the content will appear all at once like before, or progressively chunk by chunk? Why?
  • Click Read File.

The file text appears almost instantly. For a small file, it arrives in one or two chunks so it looks immediate. The critical difference is the first byte arrives within milliseconds instead of after a multi-second wait. The --- Stream complete --- marker confirms the stream finished cleanly.

Why does my small file arrive in one chunk?

Node.js reads up to 64 KB per chunk by default. Your sample file is well under that size, so the entire file fits in a single chunk. With larger files (or when a Transform stream adds processing time), you would see multiple chunks arrive one by one. You will add that processing in the next step.

✔️ Awesome, I've got everything!

Great. Double check you have saved both server.js and index.html.

ⓧ I'd like to double check the full code

Here is the full server.js at the end of this step:

const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');

const UPLOAD_DIR = path.join(__dirname, 'uploads');
const PORT = 3000;

if (!fs.existsSync(UPLOAD_DIR)) {
  fs.mkdirSync(UPLOAD_DIR);
}

function getUploadedFilePath() {
  return path.join(UPLOAD_DIR, 'uploaded.txt');
}
const server = http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }

  if (req.method === 'POST' && req.url === '/upload') {
    const filePath = getUploadedFilePath();
    const chunks = [];
    req.on('data', (chunk) => chunks.push(chunk));
    req.on('end', () => {
      const buffer = Buffer.concat(chunks);
      fs.writeFileSync(filePath, buffer);
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end(`Upload complete! ${buffer.length} bytes received.`);
    });
    req.on('error', (err) => {
      res.writeHead(500);
      res.end(`Upload error: ${err.message}`);
    });
    return;
  }
  if (req.method === 'GET' && req.url === '/read') {
    const filePath = getUploadedFilePath();
    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet.');
      return;
    }
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    const readStream = fs.createReadStream(filePath);
    readStream.pipe(res);
    return;
  }

  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

And here is the full index.html:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>File Stream Processor</title>
  <style>
    body { font-family: monospace; max-width: 800px; margin: 40px auto; padding: 0 20px; }
    .section { margin: 20px 0; padding: 15px; border: 1px solid #ddd; border-radius: 4px; }
    button { padding: 8px 16px; margin: 5px; cursor: pointer; }
    #output { background: #1e1e1e; color: #0f0; padding: 15px; min-height: 200px; max-height: 500px; overflow-y: auto; white-space: pre-wrap; word-wrap: break-word; border-radius: 4px; }
    .status { color: #666; font-style: italic; }
  </style>
</head>
<body>
  <h1>File Stream Processor</h1>
  <div class="section">
    <h2>1. Upload a Text File</h2>
    <input type="file" id="fileInput" accept=".txt,.csv,.log,.md">
    <button onclick="uploadFile()">Upload</button>
  </div>
  <div class="section">
    <h2>2. Read File (Streamed)</h2>
    <button onclick="readFileStreamed()">Read File</button>
  </div>
  <div class="section">
    <h2>Output</h2>
    <div id="output"><span class="status">Waiting for action...</span></div>
  </div>
  <script>
    const output = document.getElementById('output');
    function clearOutput() { output.textContent = ''; }
    async function uploadFile() {
      const file = document.getElementById('fileInput').files[0];
      if (!file) { output.textContent = 'Please select a file first.'; return; }
      clearOutput();
      output.textContent = 'Uploading...';
      const response = await fetch('/upload', { method: 'POST', body: file });
      output.textContent = await response.text();
    }
    async function readFileStreamed() {
      clearOutput();
      output.textContent = 'Reading file (streamed)...\n\n';
      const response = await fetch('/read');
      if (!response.ok) { output.textContent += await response.text(); return; }
      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        output.textContent += decoder.decode(value, { stream: true });
      }
      output.textContent += '\n--- Stream complete ---';
    }
  </script>
</body>
</html>

Your server now streams data the moment it reads it from disk. But the raw file content is not very useful on its own. Next up, you will build a Transform stream that processes each line on the fly, adding line numbers and word counts as data flows through the pipeline.

Add a Transform Stream and pipeline()

Raw file content streaming through is useful, but not very interesting. To do something meaningful with data as it flows, you need to process each piece without waiting for the entire file to load into memory.

A Transform stream sits between the source and the destination, modifying data on the fly. Combined with pipeline() for automatic error handling and backpressure, this gives you production-grade stream composition in a single function call.

In this step, get ready to:
  • Build a custom Transform stream that annotates each line with its number and word count.
  • Replace the direct pipe with pipeline() for safe, automatic stream wiring.
  • Watch annotated lines stream into the browser progressively.
Create a custom Transform stream that adds line numbers and word counts

Chunks from fs.createReadStream() do not align with line boundaries. A single chunk might contain three full lines and half of a fourth. Your Transform needs to buffer that leftover partial line and only emit complete, annotated lines.

  • Stop the running server by pressing Ctrl+C in your terminal.
  • In server.js, find the first three require lines at the top of the file:
const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');
  • Add two new imports directly below those lines, so the top of your file now looks like this:
const http = require('node:http');
const fs = require('node:fs');
const path = require('node:path');
const { Transform } = require('node:stream');
const { pipeline } = require('node:stream/promises');

What do these imports do?

Transform is the class you will use to create a stream that modifies data as it passes through. It receives chunks in, processes them, and pushes modified chunks out.

pipeline from node:stream/promises wires multiple streams together and returns a Promise. It automatically destroys all streams on error and respects backpressure.

  • Now add the createLineAnnotator function below the getUploadedFilePath function by pasting this code:
function createLineAnnotator() {
  let lineNumber = 0;
  let leftover = '';

  return new Transform({
    transform(chunk, encoding, callback) {
      // Combine leftover from previous chunk with new data
      const text = leftover + chunk.toString();
      const lines = text.split('\n');
      // Last element may be incomplete - save it for next chunk
      leftover = lines.pop();

      for (const line of lines) {
        lineNumber++;
        const wordCount = line.trim().split(/\s+/).filter(Boolean).length;
        this.push(`[Line ${lineNumber} | ${wordCount} words] ${line}\n`);
      }
      callback();
    },
    flush(callback) {
      // Emit any remaining partial line when stream ends
      if (leftover) {
        lineNumber++;
        const wordCount = leftover.trim().split(/\s+/).filter(Boolean).length;
        this.push(`[Line ${lineNumber} | ${wordCount} words] ${leftover}\n`);
      }
      callback();
    },
  });
}

What does this code do?

transform(chunk, encoding, callback) runs every time a new chunk arrives. It splits the text on newlines, but the last element might be an incomplete line. That partial piece is stored in leftover and prepended to the next chunk.

For each complete line, this.push() sends the annotated version downstream. The annotation format is [Line N | W words] original text.

flush(callback) runs once when the input stream ends. It emits any remaining partial line that never received a trailing newline.

  • Save server.js.

Seeing a syntax error on save?

Make sure the function sits between the closing } of getUploadedFilePath and the const server = http.createServer(...) line. If it is inside another function, you will get unexpected token errors.

Help me fix a syntax error in my Transform stream code.

Wire the pipeline into your server

Using await pipeline() requires the server handler to be an async function. You will now replace the entire server handler with a refactored version that uses URL parsing and streams throughout.

  • In server.js, select everything from const server = http.createServer((req, res) => { down to the }); that closes the handler (the line just above server.listen).
  • Delete that entire selection and replace it with this code that sets up the async handler and serves the HTML page:
const server = http.createServer(async (req, res) => {
  // Parse URL for clean pathname matching
  const url = new URL(req.url, `http://${req.headers.host}`);

  if (req.method === 'GET' && url.pathname === '/') {
    const htmlPath = path.join(__dirname, 'index.html');
    const htmlStream = fs.createReadStream(htmlPath);
    res.writeHead(200, { 'Content-Type': 'text/html' });
    htmlStream.pipe(res);
    return;
  }
  • Directly below that code, add the rewritten upload handler that streams progress back to the client:
  if (req.method === 'POST' && url.pathname === '/upload') {
    const filePath = getUploadedFilePath();
    const writeStream = fs.createWriteStream(filePath);
    const totalBytes = parseInt(req.headers['content-length'], 10) || 0;
    let receivedBytes = 0;

    req.on('data', (chunk) => {
      receivedBytes += chunk.length;
      writeStream.write(chunk);
      if (totalBytes > 0) {
        const percent = ((receivedBytes / totalBytes) * 100).toFixed(1);
        res.write(`Progress: ${receivedBytes}/${totalBytes} bytes (${percent}%)\n`);
      }
    });

    req.on('end', () => {
      writeStream.end();
      res.write(`\nUpload complete! ${receivedBytes} bytes received.\n`);
      res.end();
    });

    req.on('error', (err) => {
      writeStream.destroy();
      res.writeHead(500);
      res.end(`Upload error: ${err.message}`);
    });

    res.writeHead(200, { 'Content-Type': 'text/plain' });
    return;
  }

Why rewrite the upload handler?

The old upload buffered everything into memory before writing to disk. This new version uses fs.createWriteStream() to write chunks directly to the file as they arrive. It also streams progress percentages back to the browser using res.write() as each chunk lands.

This is the same streaming principle you are applying to the read endpoint. Data flows through without piling up in memory.

  • Now add the /read endpoint with pipeline(), the 404 fallback, and the closing bracket. Paste this directly below the upload handler:
  if (req.method === 'GET' && url.pathname === '/read') {
    const filePath = getUploadedFilePath();

    if (!fs.existsSync(filePath)) {
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('No file uploaded yet. Upload a file first.');
      return;
    }

    res.writeHead(200, { 'Content-Type': 'text/plain' });

    const readStream = fs.createReadStream(filePath, { encoding: 'utf8' });
    const lineAnnotator = createLineAnnotator();

    try {
      await pipeline(readStream, lineAnnotator, res);
    } catch (err) {
      if (!res.headersSent) {
        res.writeHead(500, { 'Content-Type': 'text/plain' });
      }
      res.end(`\nStream error: ${err.message}`);
    }
    return;
  }

  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('Not found');
});

What does pipeline() do here?

await pipeline(readStream, lineAnnotator, res) wires three streams together: the file ReadStream feeds into your Transform, which feeds into the HTTP response. Data flows from disk to client with annotations added in the middle.

If any stream errors (file deleted mid-read, client disconnects), pipeline() automatically destroys all streams in the chain. You do not need to manually remove listeners or close handles. It also handles backpressure: if the client reads slowly, the file read pauses automatically.

  • Save server.js and start the server by running:
node server.js
  • Open http://localhost:3000 in your browser. You should see the File Stream Processor page load.

Server won't start?

Check for mismatched braces. The const server = http.createServer(async (req, res) => { line must have a matching }); before server.listen. Count your opening and closing braces in the handler.

Help me debug why my Node.js server won't start after refactoring the handler.

Update the browser and test the full flow

The upload endpoint now streams progress back instead of a single response. The browser code needs to read that streamed response using response.body.getReader() the same way it reads the file stream.

  • In index.html, find this heading text:
<h2>2. Read File (Streamed)</h2>
  • Change it to include "with Annotations":
<h2>2. Read File (Streamed with Annotations)</h2>
  • Next, find the <style> block. Add this rule after the body { ... } line:
h1 { color: #333; }
  • Now replace the entire uploadFile function inside the <script> tag with this version that streams the upload response:
    async function uploadFile() {
      const fileInput = document.getElementById('fileInput');
      const file = fileInput.files[0];
      if (!file) {
        output.textContent = 'Please select a file first.';
        return;
      }

      clearOutput();
      output.textContent = 'Uploading...\n';

      const response = await fetch('/upload', {
        method: 'POST',
        headers: { 'Content-Length': file.size },
        body: file,
      });

      const reader = response.body.getReader();
      const decoder = new TextDecoder();

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        output.textContent += decoder.decode(value, { stream: true });
      }
    }

What changed in the upload function?

The old version used await response.text() which waits for the entire response before displaying anything. The new version uses response.body.getReader() to read chunks as they arrive, so you see upload progress percentages tick up in real time.

This is the same streaming read pattern the readFileStreamed function already uses.

  • Save index.html.
  • Refresh http://localhost:3000 in your browser.

Before you click anything, predict: when you click Read File, will the text appear all at once or line by line? Will it look different from before?

  • Click Read File.

You should see lines appear progressively in the output panel, each prefixed with annotations like [Line 1 | 5 words] The quick brown fox jumped.

  • Now test the upload. Click Choose File, select any .txt file from your computer, and click Upload.

You should see progress percentages stream in as the file uploads, followed by a completion message.

  • Click Read File again to see your newly uploaded file streamed back with line numbers and word counts on every line.

Lines not annotated?

If you see raw text without [Line N | W words] prefixes, the pipeline is not using your Transform. Check that the /read handler creates a lineAnnotator with createLineAnnotator() and passes it as the middle argument to pipeline().

If you see "No file uploaded yet", upload a file first using the form at the top of the page.

Help me debug why my Transform stream annotations are not appearing.

✔️ Awesome, I've got everything!

Great. Double check you have saved both server.js and index.html.

ⓧ I'd like to double check the full code

Here is the full server.js at this point. Scroll through and confirm each section matches yours:

  • Lines 1-5: Five require statements (http, fs, path, Transform, pipeline).
  • Lines 7-12: Constants, UPLOAD_DIR setup, and getUploadedFilePath function.
  • Lines 14-42: The createLineAnnotator function with transform and flush methods.
  • Lines 44-100: The async server handler with GET /, POST /upload, GET /read, and 404 fallback.
  • Lines 102-104: server.listen on port 3000.

Here is the full index.html. Confirm these sections:

  • The <style> block includes h1 { color: #333; } after the body rule.
  • Section 2 heading reads "Read File (Streamed with Annotations)".
  • The uploadFile function uses response.body.getReader() and sends a Content-Length header.
  • The readFileStreamed function is unchanged from the previous step.

Your server now processes data on the fly. A single pipeline() call wires the file, the Transform, and the HTTP response together with automatic cleanup. Next up, you will put this pattern to work on a real-world use case: searching through a file and streaming only the matching lines.

Secret mission

Add a Streaming Search Endpoint

Your streaming pipeline pattern is solid. Now apply it to a real-world scenario: build a search endpoint that filters through the uploaded file and streams only matching lines to the browser, with search terms highlighted as they flow through a second Transform stream.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The file-stream-processor folder on your Desktop (contains server.js, index.html, and the uploads/ folder).
  • A Node.js process running on port 3000.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with your stream processor.

  • Your server continues running at http://localhost:3000.
  • Your uploaded files remain in the uploads/ folder.
  • You can keep uploading and streaming files at any time.

Pause - I'll come back to this later

Stop the running server to free up your terminal and port 3000. Your project files stay on disk so you can restart any time.

  • Switch back to the terminal where your server is running.
  • Press Ctrl+C to stop the server.
  • When you are ready to resume, open a terminal in the file-stream-processor folder and run node server.js.

Delete - I don't want to use this again

Remove all project files and stop the running process. This deletes everything you built in this project.

  • Switch back to the terminal where your server is running.
  • Press Ctrl+C to stop the server.
  • Navigate to your Desktop by running:
cd ~/Desktop
  • Delete the project folder. On macOS/Linux, run:
rm -rf file-stream-processor
  • On Windows (PowerShell), run:
Remove-Item -Recurse -Force file-stream-processor
  • Confirm the folder is gone by running ls (macOS/Linux) or dir (Windows) on your Desktop. You should no longer see file-stream-processor listed.

That's a Wrap!

That's a Wrap!

Nice work! You just built a zero-dependency Node.js HTTP server that streams file data in real time using only built-in modules.

You've learned how to:

  • Experience the buffering problem firsthand (a server that goes silent while loading an entire file, then dumps everything at once) and fix it by piping a ReadStream directly to an HTTP response for instant first-byte delivery.
  • Create a custom Transform stream that processes data on the fly (line numbering, word counting) without loading the file into memory.
  • Use pipeline() from node:stream/promises for safe, production-grade stream composition with automatic error handling and backpressure.
  • Secret Mission: Build a streaming search endpoint that filters and highlights matching lines through a composed pipeline of multiple Transform streams.

Ready to quiz yourself?