Event-Driven Order Pipeline with RabbitMQ

Build two Node.js microservices that communicate through a RabbitMQ message broker.

Introduction

30 Second Summary

When you order something online, dozens of services need to coordinate behind the scenes. If one goes down or runs slowly, your order can disappear unless those services communicate through something more reliable than direct calls to each other.

In this project, you will build an event-driven order processing pipeline where two Node.js services communicate through a RabbitMQ message broker. You will POST orders, watch them travel through the broker, and observe what happens when things go wrong.

What You'll Build

You will fire off an order from your terminal, watch it flow through a hosted message broker to a separate payment service, and see it processed within milliseconds.

By the end of this project, you'll have:

  • A live pipeline where you POST an order and see it appear in the payment service's terminal, proving two independent services can exchange data through CloudAMQP without ever calling each other directly.
  • A system that survives crashes. You can kill the payment service mid-processing, restart it, and watch unacknowledged messages automatically redeliver instead of vanishing.
  • A dead-letter queue that catches failed messages so you can inspect exactly what went wrong instead of losing data silently.
  • Secret Mission: Add automatic retry with exponential backoff that re-attempts failed messages up to 3 times before dead-lettering them.

Are there any prerequisites?

This project assumes you are comfortable with Node.js, REST APIs, and basic async/await patterns. You do not need prior experience with message brokers or RabbitMQ.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters. This step is your chance to commit to the goal before any hands-on work begins.

Set Up Your Project and CloudAMQP

Your order pipeline needs two services that talk to each other without being directly connected. A message broker sits between them, accepting messages from one service and delivering them to the other.

Instead of installing RabbitMQ locally (which requires Erlang and extra configuration), you will use CloudAMQP to get a fully managed instance in seconds. In this step, you will set up your Node.js project and connect it to a free cloud-hosted RabbitMQ broker.

In this step, get ready to:
  • Confirm Node.js is installed and initialize your project with dependencies.
  • Create a free CloudAMQP instance running the "Little Lemur" plan.
  • Verify your code can connect to RabbitMQ by running a test script.
Initialize the Node.js project

Before writing any code, you need Node.js version 18 or higher installed on your machine. Both Express and amqplib (the RabbitMQ client library) require it.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Cursor and press Enter to open it.
  • Open the integrated terminal by pressing Ctrl+` (macOS and Windows).
  • Check your Node.js version by running this command:
node --version

You should see a version number printed in your terminal.

✔️ I see version 18 or higher

Your Node.js version is compatible. Continue to the next instruction below.

ⓧ I see an older version

You need Node.js 18 or higher. Visit the Node.js download page and download the latest LTS version.

After installing, close and reopen your terminal in Cursor, then run node --version again to confirm it shows 18 or higher.

macOS default version

macOS can ship an older system Node.js that shadows a newer install. If you still see an old version after installing, use nvm (Node Version Manager) to install and activate the version you need.

ⓧ Command not found

Node.js is not installed yet. Visit the Node.js download page and download the latest LTS version.

Run the installer and follow the prompts. After it finishes, close and reopen your terminal in Cursor, then run node --version to confirm you see a version number of 18 or higher.

  • Move to your Desktop and create your project folder by running these commands:
cd ~/Desktop
mkdir order-pipeline
cd order-pipeline
npm init -y

What do these commands do?

  • cd ~/Desktop moves your terminal to the Desktop so you can easily find your project folder later.
  • mkdir order-pipeline creates a new folder for the project.
  • npm init -y generates a package.json file with default settings, which tracks your project's dependencies and scripts.

You should see output ending with a JSON preview of your new package.json file.

Don't see the JSON output?

  • Make sure you ran all four commands in sequence. If your terminal shows an error after mkdir, the folder might already exist from a previous attempt. Run cd ~/Desktop/order-pipeline and then npm init -y directly.

Still stuck? Help me troubleshoot the npm init error in my terminal.

  • Install the three packages your project needs by running:
npm install express amqplib dotenv

What are these packages?

  • express is a web framework for building HTTP endpoints (your Order Service will use it).
  • amqplib is the Node.js client for connecting to RabbitMQ using the AMQP protocol.
  • dotenv loads environment variables from a .env file so you can keep secrets out of your code.

You should see output showing packages being added, ending with a line like "added X packages".

Install failed?

  • Confirm you are inside the order-pipeline folder by running pwd. You should see a path ending in order-pipeline.
  • If you see permission errors on macOS, try prefixing the command with sudo.

Need more help? Help me fix my npm install error for express, amqplib, and dotenv.

  • Open the order-pipeline folder in Cursor by clicking File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the order-pipeline folder, and click Open.

Your Cursor sidebar should now show package.json, package-lock.json, and a node_modules folder.

  • Create the two service folders by running this command in the Cursor terminal (open it again with Ctrl+` if it closed):
mkdir order-service payment-service
  • Confirm the folders exist by running:
ls

You should see node_modules, order-service, package-lock.json, package.json, and payment-service in the output.

  • Click on package.json in the Cursor sidebar to open it.
  • Select all the contents (Cmd+A on macOS or Ctrl+A on Windows) and replace them with the following:
{
  "name": "order-pipeline",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "order-service": "node order-service/index.js",
    "payment-service": "node payment-service/index.js",
    "test-connection": "node test-connection.js"
  },
  "dependencies": {
    "amqplib": "^2.0.1",
    "dotenv": "^16.4.7",
    "express": "^5.2.1"
  }
}

What did you change?

  • "type": "module" tells Node.js to use ES module import/export syntax instead of the older require() style.
  • The scripts section adds shortcut commands so you can later start each service with a simple npm run command.
  • Save the file with Cmd+S (macOS) or Ctrl+S (Windows).
  • Create a new file by clicking the new file icon in the Cursor sidebar (or press Cmd+N / Ctrl+N).
  • Name the file .env and add this single line:
AMQP_URL=your-cloudamqp-url-here
  • Save .env.

You will replace that placeholder with a real URL in the next substep.

Create a CloudAMQP account and instance

CloudAMQP hosts RabbitMQ in the cloud so you do not need to install or configure anything locally. The free "Little Lemur" plan gives you up to 100 queues and 1 million messages per month, which is plenty for this project.

  • Open your browser and go to https://customer.cloudamqp.com/signup.
  • Sign up using your Google account, GitHub account, or email.
  • If you signed up with email, confirm your email address before continuing.

Once you are logged into the CloudAMQP dashboard, you are ready to create your RabbitMQ instance.

  • Click the green Create New Instance button.
  • In the Name field, type order-pipeline.
  • For Plan, select Little Lemur (the free tier). No credit card is required.
  • Click Select Region.
  • Pick any region close to you (e.g. US-East or EU-West).
  • Click Review.
  • On the confirmation screen, click Create Instance.

Your instance provisions immediately. You should see it listed on your dashboard with a green status indicator and the "Little Lemur" plan label.

Don't see the green status?

Give it 10-15 seconds and refresh the page. Free-tier instances provision almost instantly but occasionally take a moment.

Need help? My CloudAMQP instance is not showing a green status after creating it.

  • Click on your order-pipeline instance name to open its details page.
  • Find the AMQP URL field on the details page. It starts with amqps://.
  • Copy the full URL.
  • Switch back to Cursor and open .env.
  • Replace your-cloudamqp-url-here with the URL you just copied. Your file should now look like:
AMQP_URL=amqps://your-username:your-password@your-instance.cloudamqp.com/your-username
  • Save .env.

Why not paste the URL directly in code?

The .env file keeps credentials separate from your application code. This way, you never accidentally commit secrets to Git.

Verify the connection

You have the project set up and an AMQP URL in your .env file. Now you will write a small script to prove your code can actually reach the broker.

  • Create a new file in the Cursor sidebar called test-connection.js (at the root of your order-pipeline folder, not inside either service folder).
  • Add the following code:
import "dotenv/config";
import amqplib from "amqplib";

async function testConnection() {
  try {
    // Attempt to connect using the URL from .env
    const connection = await amqplib.connect(process.env.AMQP_URL);
    console.log("Connected to RabbitMQ successfully!");
    await connection.close();
  } catch (error) {
    console.error("Failed to connect:", error.message);
  }
}

testConnection();

What does this code do?

  • import "dotenv/config" automatically loads variables from .env into process.env.
  • amqplib.connect() opens a TCP connection to your CloudAMQP broker using the AMQP_URL value.
  • If the connection succeeds, it logs a success message and immediately closes the connection to clean up.
  • If it fails (wrong URL, network issue), the catch block prints the error message so you know what went wrong.
  • Save test-connection.js.

Before you run this, what do you expect to see if your AMQP URL is correct?

  • Run the test script in your Cursor terminal:
node test-connection.js

You should see Connected to RabbitMQ successfully! printed in your terminal.

Seeing "Failed to connect"?

  • Open .env and check that the URL starts with amqps:// (note the "s" for TLS).
  • Make sure there are no extra spaces or line breaks in the URL.
  • Confirm your CloudAMQP instance shows a green status on the dashboard. If it is still provisioning, wait a few seconds and try again.
  • If you see a "getaddrinfo" error, your URL hostname might be wrong. Go back to the CloudAMQP details page and re-copy the full AMQP URL.

Still not working? Help me debug my RabbitMQ connection error from test-connection.js.

✔️ Awesome, I've got everything!

Great. Double check you have saved both package.json and test-connection.js.

ⓧ I'd like to double check the full code

Here is the full content for each file you should have at this point.

package.json:

{
  "name": "order-pipeline",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "order-service": "node order-service/index.js",
    "payment-service": "node payment-service/index.js",
    "test-connection": "node test-connection.js"
  },
  "dependencies": {
    "amqplib": "^2.0.1",
    "dotenv": "^16.4.7",
    "express": "^5.2.1"
  }
}

.env:

AMQP_URL=amqps://your-username:your-password@your-instance.cloudamqp.com/your-username

test-connection.js:

import "dotenv/config";
import amqplib from "amqplib";

async function testConnection() {
  try {
    const connection = await amqplib.connect(process.env.AMQP_URL);
    console.log("Connected to RabbitMQ successfully!");
    await connection.close();
  } catch (error) {
    console.error("Failed to connect:", error.message);
  }
}

testConnection();

Your project is wired up and talking to RabbitMQ. Next up, you will build the Order Service that publishes messages into the broker.

Build the Order Service

Your connection to CloudAMQP is verified and the broker is ready to route messages. Now you need something to actually produce those messages.

The Order Service is an Express server that accepts HTTP requests and translates them into messages on the broker. This decouples the "accept order" action from the "process payment" action, so the two services can scale and fail independently.

In this step, get ready to:
  • Set up an Express server that connects to RabbitMQ on startup.
  • Create a POST /orders endpoint that publishes order events to a durable queue.
  • Verify the full flow by POSTing an order and seeing it logged.
Set up the Express server and RabbitMQ connection

The Order Service lives in a single file that handles two jobs: connecting to RabbitMQ when the server starts, and publishing messages when an order arrives.

  • Create a new file called order-service/index.js inside your project folder.
  • Paste the following code into the file:
import "dotenv/config";
import express from "express";
import amqplib from "amqplib";
import crypto from "crypto";

const app = express();
app.use(express.json());

// Will hold the RabbitMQ channel for publishing
let channel;

// Connect to RabbitMQ and create the orders queue
async function connectRabbitMQ() {
  const connection = await amqplib.connect(process.env.AMQP_URL);
  channel = await connection.createChannel();
  await channel.assertQueue("orders", { durable: true });
  console.log("Order Service connected to RabbitMQ");
}

What does this code do?

This sets up the foundation for your Order Service.

  • import "dotenv/config" loads your .env file so AMQP_URL is available.
  • let channel stores the RabbitMQ channel at module level so the POST handler can use it later.
  • connectRabbitMQ() opens a connection, creates a channel, and asserts a durable queue named orders. Durable means the queue survives broker restarts.
Implement the POST /orders endpoint

With the connection infrastructure ready, you need a route that accepts incoming orders, wraps them as JSON messages, and pushes them onto the queue.

  • Add the following code below your connectRabbitMQ function in order-service/index.js:
// Accept incoming orders and publish to the queue
app.post("/orders", (req, res) => {
  const order = {
    orderId: req.body.orderId || crypto.randomUUID(),
    item: req.body.item,
    amount: req.body.amount,
    createdAt: new Date().toISOString(),
  };

  // Publish as a persistent JSON message
  channel.sendToQueue("orders", Buffer.from(JSON.stringify(order)), {
    persistent: true,
  });

  console.log(`Order published: ${order.orderId}`);
  res.status(201).json({ orderId: order.orderId, status: "accepted" });
});

// Start listening only after RabbitMQ connects
connectRabbitMQ().then(() => {
  app.listen(3001, () => {
    console.log("Order Service running on http://localhost:3001");
  });
});

What does this code do?

This is the publishing logic that turns an HTTP request into a message on the broker.

  • The endpoint accepts an optional orderId in the request body. If none is provided, it generates one with crypto.randomUUID().
  • channel.sendToQueue sends the order as a JSON buffer. The persistent: true option tells RabbitMQ to write the message to disk so it survives broker restarts.
  • The server only starts listening after connectRabbitMQ() resolves. This guarantees the channel exists before any request arrives.
  • Save order-service/index.js.

Why persistent messages?

Without persistent: true, messages live only in memory. If the broker restarts, every undelivered message disappears. Persistence writes messages to disk, so they survive crashes.

Before you run the server, what do you expect to see in the terminal when you start it?

  • Start the Order Service by running this command:
node order-service/index.js

You should see two lines printed:

Order Service connected to RabbitMQ
Order Service running on http://localhost:3001
  • Open a second terminal window. Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) in Cursor, type Terminal: Create New Terminal, and press Enter.
  • Send a test order by running this command:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d '{"item":"keyboard","amount":59.99}'

In the second terminal, you should see a JSON response with your orderId and status: "accepted". Switch back to the first terminal where the server is running. You should see a log line like:

Order published: <your-generated-uuid>

Not seeing the log or getting an error?

  • If you see ECONNREFUSED or a connection error, double-check that your AMQP_URL in .env is correct. Make sure you copied the full URL from CloudAMQP including the protocol amqps://.
  • If curl returns Connection refused, the server is not running. Check the first terminal for startup errors.
  • If you see a syntax error mentioning import, confirm your package.json includes "type": "module".

Still stuck? Why is my Order Service failing to start or publish messages?

  • Optionally, verify the queue exists in your CloudAMQP dashboard. Click the RabbitMQ Manager button in the top-left corner of your instance page, then select the Queues tab. You should see the orders queue listed with the D (durable) flag.

✔️ Awesome, I've got everything!

Great. Double check you have saved order-service/index.js and that your server logs Order published: when you POST an order.

ⓧ I'd like to double check the full code

Your order-service/index.js should contain these two sections in order:

  • The imports, Express setup, let channel, and connectRabbitMQ function (first code block you pasted).
  • The app.post("/orders", ...) handler and the connectRabbitMQ().then(...) startup block (second code block you pasted).

Scroll up and confirm both blocks are present, in that order. The file should start with import "dotenv/config" and end with the app.listen call inside connectRabbitMQ().then().

Your Order Service is publishing messages to RabbitMQ. But right now those messages have nowhere to go. Next up, you will build the Payment Service that consumes orders from the queue and processes them.

Build the Payment Service (Naive Consumer)

Your Order Service is publishing messages to RabbitMQ. Now you need something on the other end to pick those messages up and act on them.

In this step, you'll build a Payment Service that consumes order messages from the queue. You'll start with the simplest possible approach and see how it behaves when the consumer crashes mid-processing.

In this step, get ready to:
  • Create the Payment Service consumer.
  • Test the full order-to-payment message flow.
  • Observe what happens when the consumer dies mid-processing.
Create the consumer that reads from the orders queue

The Payment Service connects to the same orders queue your Order Service publishes to. When a message arrives, the consumer parses it and simulates a 2-second payment processing delay.

  • Create a new file called index.js inside the payment-service/ folder. Add the following code:
import "dotenv/config";
import amqplib from "amqplib";

async function start() {
  const connection = await amqplib.connect(process.env.AMQP_URL);
  const channel = await connection.createChannel();

  await channel.assertQueue("orders", { durable: true });

  console.log("Payment Service waiting for orders...");

  channel.consume(
    "orders",
    async (msg) => {
      if (!msg) return;
      const order = JSON.parse(msg.content.toString());
      console.log(`Processing order: ${order.orderId}`);
      await new Promise((resolve) => setTimeout(resolve, 2000));
      console.log(`Payment processed for order ${order.orderId}: $${order.amount}`);
    },
    { noAck: true }
  );
}

start().catch(console.error);

What does this code do?

This consumer mirrors the connection pattern from your Order Service but reads from the queue instead of writing to it.

  • channel.assertQueue("orders", { durable: true }) ensures the queue exists. Both producer and consumer assert the same queue so it doesn't matter which starts first.
  • channel.consume registers a callback that fires every time a message arrives in the queue.
  • { noAck: true } tells RabbitMQ to consider a message acknowledged the instant it is delivered to this consumer. The broker removes it from the queue immediately.
  • The 2-second setTimeout simulates real processing work (charging a credit card, calling an external API).
  • Save payment-service/index.js.
  • Start the Payment Service by running:
node payment-service/index.js

You should see Payment Service waiting for orders... printed in your terminal. The consumer is now listening for messages.

Seeing a connection error?

  • Make sure your .env file is in the project root (not inside payment-service/). The dotenv/config import looks for .env relative to where you run the command.
  • Confirm you're running the command from the project root folder (where package.json lives), not from inside payment-service/.
  • help me debug my RabbitMQ connection error in the Payment Service

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

import "dotenv/config";
import amqplib from "amqplib";

async function start() {
  const connection = await amqplib.connect(process.env.AMQP_URL);
  const channel = await connection.createChannel();

  await channel.assertQueue("orders", { durable: true });

  console.log("Payment Service waiting for orders...");

  channel.consume(
    "orders",
    async (msg) => {
      if (!msg) return;
      const order = JSON.parse(msg.content.toString());
      console.log(`Processing order: ${order.orderId}`);
      await new Promise((resolve) => setTimeout(resolve, 2000));
      console.log(`Payment processed for order ${order.orderId}: $${order.amount}`);
    },
    { noAck: true }
  );
}

start().catch(console.error);
Demonstrate the message loss problem

Your consumer is running and ready. Time to send it a real order and see how the system handles a crash during processing.

  • Stop the Payment Service with Ctrl+C (you'll restart it in a moment alongside the Order Service).
  • Open a second terminal in Cursor. You need two terminals running side by side: one for each service.
  • In the first terminal, start the Order Service by running:
node order-service/index.js
  • In the second terminal, start the Payment Service by running:
node payment-service/index.js

Both terminals should now show their respective "connected" and "waiting" messages.

  • Open a third terminal. POST an order to the Order Service by running:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d "{\"item\":\"keyboard\",\"amount\":59.99}"

Watch your terminals. In the Order Service terminal you should see Order published: [orderId]. In the Payment Service terminal you should see Processing order: [orderId] followed by Payment processed for order [orderId]: $59.99 two seconds later.

What just happened?

Your order flowed from an HTTP request, through a message broker, and into a completely separate service. The Order Service and Payment Service never talk to each other directly. RabbitMQ sits between them, decoupling sender from receiver.

That worked perfectly. Now let's see what happens when things go wrong.

  • POST another order by running the same curl command:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d "{\"item\":\"monitor\",\"amount\":299.99}"

Before you do anything else, think about this: the Payment Service logs "Processing order" and then waits 2 seconds before finishing. What do you think happens to the message if you kill the Payment Service during those 2 seconds?

  • The instant you see Processing order: appear in the Payment Service terminal, press Ctrl+C immediately to kill it. You need to act within the 2-second window before processing finishes.

The Payment Service is now dead. It crashed before it finished processing that order. In a real system, this could be a server reboot, a deployment, or an out-of-memory kill.

  • Restart the Payment Service by running:
node payment-service/index.js

You see Payment Service waiting for orders... again. But the order for the monitor never reappears. It is gone forever.

The message vanished

Because { noAck: true } tells RabbitMQ that delivery equals acknowledgment, the broker deleted the message the instant it sent it to your consumer. It doesn't matter that your code hadn't finished processing. The broker's job was done.

In a real payment system, this means a customer got charged (or worse, never got charged and received their item for free) with no record of what happened. This is the exact class of bug that noAck: true introduces.

Message still appeared after restart?

  • You may not have killed the service fast enough. The 2-second window is tight. Try again: POST a new order and press Ctrl+C the instant Processing order: appears.
  • If you see the "Payment processed" line before you can press Ctrl+C, the message already finished processing. That's fine. It still proves the point: with noAck: true, the message is gone from the queue the moment it's delivered, regardless of outcome.
  • help me understand why my message reappeared after restarting the Payment Service

You've just experienced the fundamental problem with noAck: true. Messages vanish the moment they leave the queue, whether or not your code actually finished handling them. In the next step, you'll fix this with manual acknowledgment and a dead-letter queue that catches failures instead of losing them.

Add Acknowledgment and Dead-Letter Queue

In the last step, you saw messages vanish the moment the Payment Service crashed. That is the exact failure mode that causes double charges and lost orders in production.

The fix has two parts. Manual acknowledgment tells the broker "I finished processing this" so unfinished messages automatically redeliver. A dead-letter queue catches messages that fail processing so they land somewhere visible instead of disappearing into the void.

In this step, get ready to:
  • Set up a dead-letter exchange and queue for failed messages.
  • Switch from automatic to manual acknowledgment with error handling.
  • Verify messages survive crashes and route to the DLQ on failure.
Set up the dead-letter exchange and queue

Before you write any code, you need to remove the existing orders queue from RabbitMQ. RabbitMQ refuses to change the arguments on a queue that already exists. Since you created orders without dead-letter arguments in the previous steps, you must delete it first and let the new code recreate it with the right configuration.

  • Switch to your CloudAMQP dashboard in the browser.
  • Click the RabbitMQ Manager button to open the management UI.
  • Click the Queues tab at the top.
  • Click the orders queue name.
  • Scroll to the bottom of the page and click Delete Queue.
  • Confirm the deletion. The queue disappears from the list.

Why delete the queue first?

RabbitMQ treats queue arguments as immutable after creation. Calling assertQueue with different arguments than an existing queue will throw a channel error. Deleting and recreating is the clean path.

Now you will rewrite payment-service/index.js from scratch. The new version declares a dead-letter exchange, a dead-letter queue, and binds them together before recreating the main orders queue with dead-letter routing.

  • Open payment-service/index.js in Cursor.
  • Select all the existing content and replace it with this code:
import "dotenv/config";
import amqplib from "amqplib";

const processedOrders = new Set();

async function start() {
  const connection = await amqplib.connect(process.env.AMQP_URL);
  const channel = await connection.createChannel();

  // Set up dead-letter exchange and queue
  await channel.assertExchange("orders-dlx", "direct", { durable: true });
  await channel.assertQueue("orders-dlq", { durable: true });
  await channel.bindQueue("orders-dlq", "orders-dlx", "orders");

  // Assert main queue with dead-letter configuration
  await channel.assertQueue("orders", {
    durable: true,
    arguments: {
      "x-dead-letter-exchange": "orders-dlx",
      "x-dead-letter-routing-key": "orders",
    },
  });

  // Prefetch 1 message at a time
  await channel.prefetch(1);

  console.log("Payment Service waiting for orders...");

What does this code do?

assertExchange("orders-dlx", "direct", { durable: true }) creates a dead-letter exchange. This is a special exchange where rejected messages get routed.

assertQueue("orders-dlq", { durable: true }) creates the dead-letter queue that holds failed messages for inspection. bindQueue connects the two so messages routed to orders-dlx with routing key orders land in orders-dlq.

The orders queue now includes x-dead-letter-exchange and x-dead-letter-routing-key arguments. Any message that gets rejected will automatically route through this exchange into the DLQ.

prefetch(1) ensures the consumer only takes one message at a time. This prevents a single slow consumer from hoarding messages while others sit idle.

Switch to manual acknowledgment with error handling

The consumer now needs to explicitly tell the broker when it finishes processing. On success, it acknowledges. On failure, it rejects the message and the broker routes it to the dead-letter queue.

  • Directly below the console.log("Payment Service waiting for orders..."); line, add the consumer logic:
  channel.consume(
    "orders",
    async (msg) => {
      if (!msg) return;

      const order = JSON.parse(msg.content.toString());

      try {
        // Simulate failure for testing
        if (order.item === "FAIL") {
          throw new Error("Payment processing failed");
        }

        // Simulate processing delay
        await new Promise((resolve) => setTimeout(resolve, 2000));

        console.log(
          `Payment processed for order ${order.orderId}: $${order.amount}`
        );
        channel.ack(msg);
      } catch (error) {
        console.error(
          `Processing failed for order ${order.orderId}: ${error.message}`
        );
        // Nack without requeue - sends to dead-letter queue
        channel.nack(msg, false, false);
      }
    },
    { noAck: false }
  );
}

start().catch(console.error);

What does this code do?

{ noAck: false } switches to manual acknowledgment. The broker holds the message until your code explicitly confirms it.

channel.ack(msg) tells the broker "processing succeeded, remove this message." If the consumer crashes before calling ack, the broker redelivers the message to another consumer (or the same one after restart).

channel.nack(msg, false, false) rejects the message without requeueing it. The second false means "do not put this back in the orders queue." Instead, the dead-letter arguments route it to orders-dlq.

The if (order.item === "FAIL") check is an intentional failure trigger for testing. You will use it to prove messages route to the DLQ.

  • Save payment-service/index.js.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

Your payment-service/index.js should contain these two sections in order:

  • The imports, processedOrders Set, start() function opening, dead-letter exchange/queue setup, main queue assertion with DLX arguments, prefetch(1), and the "waiting for orders" log (first code block above).
  • The channel.consume call with the try/catch, ack/nack logic, { noAck: false }, closing brace, and start().catch(console.error) (second code block above).

Scroll up and confirm both code blocks are present in your file, in order, with no leftover code from the old version.

Verify message durability and dead-letter routing

Time to prove both fixes work. First, confirm that messages survive a crash. Then, trigger an intentional failure and watch the message land in the dead-letter queue.

  • Start the Order Service by running:
node order-service/index.js
  • Open a second terminal and start the Payment Service by running:
node payment-service/index.js

You should see "Payment Service waiting for orders..." in the second terminal.

Seeing a channel error about inequivalent arguments?

This means the old orders queue still exists on your CloudAMQP instance. Return to the RabbitMQ Manager UI, navigate to Queues, click orders, scroll down, and click Delete Queue. Then restart the Payment Service.

Help me fix this channel error

  • In a third terminal, post an order by running:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d '{"item":"keyboard","amount":59.99}'
  • You will see "Processing order..." begin in the Payment Service terminal. While it is still processing (within the 2-second delay), press Ctrl+C in the Payment Service terminal to kill it.
  • Restart the Payment Service by running:
node payment-service/index.js

The message redelivers automatically. You will see the Payment Service pick it up again and log the payment processed message after reconnecting. The message no longer vanishes.

Why does this work now?

With noAck: false, the broker holds the message until channel.ack(msg) is called. When you killed the process before ack ran, the broker detected the broken connection and put the message back in the queue for the next consumer.

Now test the dead-letter routing. Before you run this, what do you think will happen when the Payment Service receives an order with item set to "FAIL"?

  • Send a FAIL order by running:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d '{"item":"FAIL","amount":0}'

In the Payment Service terminal, you will see a log like: Processing failed for order [orderId]: Payment processing failed. The message was rejected and routed to the dead-letter queue.

  • Switch to your CloudAMQP RabbitMQ Manager UI in the browser.
  • Click the Queues tab.
  • You should see orders-dlq listed with 1 message ready.

That message is safely parked. It did not disappear. You can inspect it, replay it later, or alert on it. This is the dead-letter pattern in action.

Don't see orders-dlq in the queue list?

Make sure the Payment Service started without errors. The DLQ is only created when the Payment Service connects and runs assertQueue("orders-dlq"). Check your terminal for error messages.

Help me debug my dead-letter queue

Messages no longer vanish. Crashes trigger automatic redelivery, and failures route to a visible dead-letter queue. Next up, you will handle the other side of the reliability coin: what happens when the same message arrives twice.

Implement Idempotent Consumer

Your pipeline now handles failures gracefully. Messages that crash the consumer redeliver automatically, and messages that fail processing route to the dead-letter queue.

There is still one gap. In distributed systems, messages can be delivered more than once. A network hiccup, or a consumer crash after processing but before acknowledging, causes the broker to redeliver the same message. Without protection, that means double charges and corrupted data.

In this step, get ready to:
  • Add an idempotency check that detects duplicate orders before processing.
  • Test duplicate handling by sending the same order twice and confirming it only processes once.
Add the idempotency check before processing

An idempotent consumer produces the same result no matter how many times the same message arrives. The pattern is simple: before processing an order, check whether you have already seen its ID.

Your payment-service/index.js already declares a processedOrders Set at the top of the file. You will now wire it into the consume callback.

  • In payment-service/index.js, find this line inside the consume callback:
const order = JSON.parse(msg.content.toString());
  • Directly below that line (before the try block), add the following idempotency check:
      // Idempotency check
      if (processedOrders.has(order.orderId)) {
        console.log(`Duplicate order ${order.orderId}, skipping`);
        channel.ack(msg);
        return;
      }

What does this code do?

Before any processing happens, the consumer checks whether it has already seen this orderId. If it has, it logs a skip message, acknowledges the message (so the broker stops redelivering it), and returns early.

Acknowledging duplicates is important. If you ignored the message without acking, the broker would keep trying to deliver it forever.

The check alone is not enough. You also need to record each successfully processed order in the Set so the check works on subsequent deliveries.

  • In the same file, find the success log inside the try block:
        console.log(
          `Payment processed for order ${order.orderId}: $${order.amount}`
        );
  • Add this line directly above that console.log:
        processedOrders.add(order.orderId);

Why add the orderId before acking?

The order matters. You add the ID to the Set, log the success, then acknowledge. If the consumer crashes between processing and acking, the broker redelivers the message. The Set catches it as a duplicate on the second attempt, preventing double-processing.

  • Save payment-service/index.js.

Why an in-memory Set?

This is a teaching simplification. The Set lives in process memory, so it resets if you restart the service. In production, you would use Redis or a database table for deduplication so processed IDs survive restarts.

The pattern itself is what matters here. Swap the Set for a Redis SISMEMBER call and the logic stays identical.

Test duplicate order handling

Your Order Service already accepts a client-provided orderId in the request body. If you include one, it uses that ID instead of generating a random one. This lets you send the exact same order twice to test idempotency.

  • Stop both services if they are running (press Ctrl+C in each terminal).
  • Start the Payment Service in one terminal by running:
node payment-service/index.js
  • Start the Order Service in a second terminal by running:
node order-service/index.js

Now you will send two orders with the same orderId. Before you do, think about what you expect to happen. If you send the same order twice, should the Payment Service process it twice, or catch the duplicate?

  • In a third terminal, send the first order by running:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d "{\"orderId\":\"test-123\",\"item\":\"keyboard\",\"amount\":59.99}"
  • Wait for the Payment Service terminal to show "Payment processed", then send the same order again by running the exact same command:
curl -X POST http://localhost:3001/orders -H "Content-Type: application/json" -d "{\"orderId\":\"test-123\",\"item\":\"keyboard\",\"amount\":59.99}"
  • Switch to the Payment Service terminal and check the logs.

You should see exactly one "Payment processed" log for the first delivery, and "Duplicate order test-123, skipping" for the second. The consumer detected the repeat and refused to process it again.

Seeing two 'Payment processed' logs?

  • Make sure you added processedOrders.add(order.orderId) inside the try block, above the success log. Without it, the Set stays empty and the check never catches anything.
  • Confirm the idempotency check is placed before the try block, not inside it. It needs to run before any processing starts.
  • Verify both curl commands include the same "orderId":"test-123" field. Without an explicit orderId, the Order Service generates a unique one each time.

Why is my Payment Service processing duplicate orders?

✔️ Awesome, I've got everything!

Great. Double check you have saved payment-service/index.js and confirmed the duplicate detection works.

ⓧ I'd like to double check the full code

Here is the complete payment-service/index.js with the idempotency logic in place. Scroll up and confirm the idempotency check block is between the JSON parse and the try block, and that processedOrders.add() is inside the try block before the success log.

import "dotenv/config";
import amqplib from "amqplib";

const processedOrders = new Set();

async function start() {
  const connection = await amqplib.connect(process.env.AMQP_URL);
  const channel = await connection.createChannel();
  // Set up dead-letter exchange and queue
  await channel.assertExchange("orders-dlx", "direct", { durable: true });
  await channel.assertQueue("orders-dlq", { durable: true });
  await channel.bindQueue("orders-dlq", "orders-dlx", "orders");

  // Assert main queue with dead-letter configuration
  await channel.assertQueue("orders", {
    durable: true,
    arguments: {
      "x-dead-letter-exchange": "orders-dlx",
      "x-dead-letter-routing-key": "orders",
    },
  });
  // Prefetch 1 message at a time
  await channel.prefetch(1);

  console.log("Payment Service waiting for orders...");

  channel.consume(
    "orders",
    async (msg) => {
      if (!msg) return;

      const order = JSON.parse(msg.content.toString());

      // Idempotency check
      if (processedOrders.has(order.orderId)) {
        console.log(`Duplicate order ${order.orderId}, skipping`);
        channel.ack(msg);
        return;
      }
      try {
        // Simulate failure for testing
        if (order.item === "FAIL") {
          throw new Error("Payment processing failed");
        }

        // Simulate processing delay
        await new Promise((resolve) => setTimeout(resolve, 2000));

        processedOrders.add(order.orderId);
        console.log(
          `Payment processed for order ${order.orderId}: $${order.amount}`
        );
        channel.ack(msg);
      } catch (error) {
        console.error(
          `Processing failed for order ${order.orderId}: ${error.message}`
        );
        // Nack without requeue - sends to dead-letter queue
        channel.nack(msg, false, false);
      }
    },
    { noAck: false }
  );
}

start().catch(console.error);

Your pipeline now handles every failure mode in the playbook: lost messages (manual ack), poisoned messages (dead-letter queue), and duplicate deliveries (idempotent consumer). Next up, you will push this further in the Secret Mission by adding automatic retries with exponential backoff.

Secret mission

Add Automatic Retry with Exponential Backoff

Your pipeline dead-letters messages on the first failure. But many failures are transient. Network blips, temporary database locks, momentary service outages. Can you add automatic retries with exponential backoff so messages get multiple chances before giving up?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on a free CloudAMQP instance and local files. There are no ongoing costs, but you can clean up if you want a fresh slate.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • CloudAMQP "Little Lemur" instance (free, no charges).
  • Local project folder order-pipeline on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the project.

  • Your CloudAMQP Little Lemur instance stays free indefinitely.
  • Queues that sit idle for more than 28 days are automatically deleted by CloudAMQP, but you can recreate them at any time.
  • Your local order-pipeline folder stays on your Desktop until you manually remove it.

Pause - I'll come back to this later

Shut down your running services to free up terminal sessions. Your CloudAMQP instance stays free and your code stays on disk.

  • In each terminal running a service (Order Service and Payment Service), press Ctrl+C to stop the process.
  • Close any unused terminal windows.
  • Your CloudAMQP instance remains active at no cost. Idle queues auto-delete after 28 days but can be recreated by running your services again.

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone.

  • Stop any running services by pressing Ctrl+C in each terminal.
  • Delete your local project folder by running:

macOS

cd ~/Desktop
rm -rf order-pipeline

Windows

cd ~\Desktop
rmdir /s /q order-pipeline
  • Delete your CloudAMQP instance:
  • Navigate to https://customer.cloudamqp.com/instance in your browser.
  • Click Edit next to your instance.
  • Select Delete to permanently remove the instance and all its data.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-grade event-driven pipeline with two independent microservices communicating through a message broker.

You've learned how to:

  • Build two independent Node.js services that communicate asynchronously through RabbitMQ, decoupling producers from consumers.
  • Implement manual acknowledgment and a dead-letter queue that catches failed messages instead of losing them into the void.
  • Build an idempotent consumer that detects duplicate messages and skips them, preventing double-processing.
  • Secret Mission: Added automatic retry with exponential backoff that gives transient failures multiple chances before dead-lettering.

Ready to quiz yourself?