Load Test and Tune a Next.js API

Write k6 load tests, stream metrics to Grafana, and fix bottlenecks.

Introduction

30 Second Summary

Every app feels fast when you're the only one using it. The real question is what happens when hundreds of users hit your API at the same time. Without testing under load, you're guessing whether your system can handle real traffic.

In this project, you will write load test scripts that simulate realistic traffic against your Next.js API, stream the results into a live dashboard, identify performance bottlenecks, and prove measurable improvements through targeted optimizations.

What You'll Build

A production-style performance engineering workflow that uses k6 to load test your Next.js API and streams results into your existing Prometheus and Grafana observability stack.

By the end of this project, you'll have:

  • A repeatable load test suite with automated pass/fail thresholds on p95/p99 latency and error rates.
  • A real-time Grafana dashboard showing k6 metrics streamed via Prometheus remote write, correlated with your application metrics.
  • Documented before-and-after proof of performance improvements from optimizations you applied based on bottleneck analysis.
  • Secret Mission: Run a breaking point test that ramps traffic until your API fails, discovering your system's exact capacity ceiling.

Are there any prerequisites?

This project assumes you already have a running Next.js API with PostgreSQL, Prometheus, and Grafana configured. You also need basic JavaScript knowledge.

If you don't have this setup yet, complete the earlier RunLayer projects in this series first.

Install k6 and Configure Prometheus Remote Write

You have a Next.js API running with Prometheus collecting metrics and Grafana displaying them. Now you need a tool that can simulate realistic traffic so you can measure how your API performs under load.

In this step, you will install k6, a load testing tool built by Grafana Labs, and connect it to your existing Prometheus instance using remote write. This lets k6 stream test metrics directly into Prometheus so you can visualize them in Grafana alongside your application metrics.

In this step, get ready to:
  • Install k6 and verify it runs on your machine.
  • Enable the Prometheus remote write receiver to accept pushed metrics.
  • Test the full k6 to Prometheus pipeline with a minimal load test.
Install k6 and verify

k6 is an open-source load testing tool with its own Go-based JavaScript runtime. You write test scripts in JavaScript, but k6 executes them in a high-performance engine designed specifically for generating HTTP traffic.

Why k6 instead of other load testing tools?

k6 has native Prometheus remote write support, which means your load test metrics flow directly into the observability stack you already have running. It also supports multi-scenario scripted tests with built-in pass/fail thresholds, so you get automated quality gates without extra tooling.

  • Install k6 using Homebrew by running this command:
brew install k6
  • Verify the installation by running:
k6 version

Check the output in your terminal.

✔️ I see version 2.0.0 or higher

Your output shows something like k6 v2.0.0 (go1.x.x, darwin/arm64). k6 is ready to go.

ⓧ I see an older version

If you see a version lower than v2.0.0, you have an outdated k6 installation. This project uses features available in v2.0.0 or later.

  • Upgrade k6 by running:
brew upgrade k6
  • Run k6 version again to confirm you now see v2.0.0 or higher.

ⓧ Command not found

If your terminal says the command is not found, Homebrew may not have linked k6 correctly.

  • Make sure Homebrew itself is working by running:
brew --version
  • If Homebrew is not installed, follow the instructions at https://brew.sh to install it first.
  • Once Homebrew is working, re-run brew install k6 and then k6 version to verify.
Enable Prometheus remote write receiver

By default, Prometheus only scrapes metrics from targets. For k6 to push metrics into Prometheus, you need to enable the remote write receiver on your Prometheus instance.

What is Prometheus remote write?

Remote write is a protocol that lets external tools push time-series data into Prometheus. When you enable the receiver, Prometheus exposes an endpoint at /api/v1/write that accepts incoming metrics.

k6 uses this endpoint to stream load test metrics in real time, so they appear alongside your application metrics without any additional infrastructure.

  • Open your Docker Compose file (or the script you use to start Prometheus).
  • Find the Prometheus service's command section.
  • Add the --web.enable-remote-write-receiver flag to the list of command arguments.

Your Prometheus command arguments should include the new flag like this:

command:
  - '--config.file=/etc/prometheus/prometheus.yml'
  - '--web.enable-remote-write-receiver'

What does this flag do?

The --web.enable-remote-write-receiver flag tells Prometheus to listen for incoming metric data at /api/v1/write. Without this flag, that endpoint returns a 404 and k6 metrics have nowhere to go.

  • Restart your Prometheus container to apply the change by running:
docker compose restart prometheus
  • Verify the remote write receiver is active by running:
curl -X POST http://localhost:9090/api/v1/write

Check the response in your terminal.

✔️ I see an error about snappy or decoding

A response mentioning snappy: corrupt input or a protobuf decoding error means the receiver is enabled. Prometheus expects properly encoded data, so sending a raw POST triggers this error. The important thing is that it did not return a 404.

ⓧ I see a 404 response

A 404 means the remote write receiver is not active.

  • Double check that --web.enable-remote-write-receiver is in your Prometheus command arguments.
  • Make sure you fully restarted the Prometheus container after adding the flag.
  • Check for startup errors by running:
docker compose logs prometheus

Prometheus won't start after adding the flag?

Make sure the flag is on its own line in the command list and starts with -- (two dashes). A common mistake is putting it inside the config file path string instead of as a separate argument.

help me debug my Prometheus remote write receiver setup

Test the k6 to Prometheus pipeline

You have k6 installed and Prometheus ready to receive metrics. Now you will run a minimal 5-second test to confirm the entire pipeline works end to end.

  • Run this command to execute a quick inline test that sends one virtual user to your API's health endpoint for 5 seconds, streaming results to Prometheus:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw --duration 5s --vus 1 \
  <(echo 'import http from "k6/http"; export default function() { http.get("http://localhost:3000/api/health"); }')

What does this command do?

K6_PROMETHEUS_RW_SERVER_URL tells k6 where to push metrics. K6_PROMETHEUS_RW_TREND_STATS controls which percentile stats become individual Prometheus time series (p95, p99, avg, min, max).

The -o experimental-prometheus-rw flag activates the Prometheus remote write output module. The inline script sends a single GET request per iteration to your health endpoint.

  • Wait for the test to finish (about 5 seconds).
  • Open Prometheus in your browser at http://localhost:9090.
  • In the query input field, type k6_ and check if autocomplete suggestions appear with k6 metric names.

✔️ I see k6_ metrics in Prometheus

Your pipeline is working. k6 successfully pushed load test metrics into Prometheus via remote write. You should see metrics like k6_http_req_duration_p95 and k6_http_reqs in the suggestions.

ⓧ No k6_ metrics appear

If no metrics with the k6_ prefix appear, the pipeline is not connected properly.

  • Check that the k6 terminal output did not show connection errors.
  • Verify Prometheus is running on port 9090 by opening http://localhost:9090 in your browser.
  • Confirm the remote write receiver is enabled by re-running the curl test from the previous substep.
  • Wait 10-15 seconds after the test finishes, then refresh Prometheus. k6 pushes metrics every 5 seconds by default.

Getting connection refused errors in k6 output?

Make sure your Next.js API is running on http://localhost:3000 and the /api/health endpoint responds. If k6 can reach the API but not Prometheus, check that Prometheus port 9090 is accessible from your host machine (not just inside Docker's network).

help me debug k6 connection errors to Prometheus remote write

Your load testing pipeline is connected. Next up, you will write a proper baseline test script that measures your API's performance floor with zero contention.

Write and Run a Baseline Smoke Test

Your k6 installation is confirmed and Prometheus is accepting remote write data from the pipeline you tested in Step 1.

Before you can identify performance problems, you need to know what "normal" looks like for your API. A baseline smoke test measures performance with a single virtual user and zero contention, giving you a performance floor to compare against when you add realistic load later.

In this step, get ready to:
  • Write a baseline smoke test script targeting your API.
  • Run the test with Prometheus output to stream metrics in real time.
  • Record your API's performance floor as a reference for future comparisons.
Create the baseline test script

A smoke test uses the lightest possible load to confirm your API responds correctly. With only 1 virtual user, any latency you measure is pure response time with no queuing or contention.

  • Create a new file called baseline.js in your project directory.
  • Add the imports and test configuration by pasting this code:
import http from 'k6/http';
import { check, sleep } from 'k6';

// Test configuration: ramp to 1 VU, hold, then ramp down
export const options = {
  stages: [
    { duration: '5s', target: 1 },
    { duration: '20s', target: 1 },
    { duration: '5s', target: 0 },
  ],
  // Automated pass/fail thresholds
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

What does this code do?

The stages array defines a traffic shape: ramp up to 1 virtual user over 5 seconds, hold that single user for 20 seconds of steady requests, then ramp back down to 0.

The thresholds object sets automated pass/fail criteria. p(95)<500 means the test fails if more than 5% of requests take longer than 500ms. rate<0.01 means the test fails if more than 1% of requests return non-2xx status codes.

  • Below the options configuration, add the test function that makes requests to your API:
export default function () {
  // Send a GET request to the recommendations endpoint
  const res = http.get('http://localhost:3000/api/recommendations');

  // Validate the response is correct, not just fast
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response body is not empty': (r) => r.body.length > 0,
  });

  // Pause between iterations to simulate realistic pacing
  sleep(1);
}

What does this code do?

The export default function is what each virtual user executes repeatedly. It sends a GET request to your recommendations endpoint and validates the response.

The check() function asserts correctness. A fast response that returns a 500 error is still a problem, so you validate both status code and body content.

The sleep(1) call pauses for 1 second between iterations. Without this, k6 would hammer your API as fast as possible, which does not represent real user behavior.

  • Save baseline.js.

File not saving?

Make sure you created baseline.js in the same directory where you will run your k6 commands. If the file is in a subfolder, the run command will not find it.

help me figure out where to save my baseline.js file

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before continuing.

ⓧ I'd like to double check the full code

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '5s', target: 1 },
    { duration: '20s', target: 1 },
    { duration: '5s', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('http://localhost:3000/api/recommendations');

  check(res, {
    'status is 200': (r) => r.status === 200,
    'response body is not empty': (r) => r.body.length > 0,
  });

  sleep(1);
}
Run the baseline and record results

With the script written, you can now execute the test. The command sends metrics to Prometheus in real time while also printing a summary to your terminal.

  • Make sure your Next.js API is running on localhost:3000 before proceeding.
  • Run the baseline test with Prometheus output by running this command:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw baseline.js

What does this command do?

K6_PROMETHEUS_RW_SERVER_URL tells k6 where to push metrics. This points to your local Prometheus remote write endpoint.

K6_PROMETHEUS_RW_TREND_STATS controls which statistics k6 exposes as separate Prometheus time series. Setting it to p(95),p(99),avg,min,max creates metrics like k6_http_req_duration_p95 and k6_http_req_duration_p99 in Prometheus.

The -o experimental-prometheus-rw flag activates the Prometheus remote write output module, streaming metrics every 5 seconds during the test.

The test takes about 30 seconds to complete. You should see a THRESHOLDS section at the top of the output with ✓ marks next to each threshold, followed by a TOTAL RESULTS section showing your HTTP metrics.

  • Record the following values from the http_req_duration line in the HTTP section: avg, p(90), and p(95).
  • Record the http_req_failed rate (should be 0.00% for a healthy baseline).

What should I see?

Your terminal output shows a THRESHOLDS section with pass/fail indicators, then TOTAL RESULTS with checks and HTTP metrics. The http_req_duration line shows aggregated latency values like avg, min, med, max, p(90), and p(95). All values should be well under 500ms for a healthy local API.

Both thresholds should show ✓ marks. If either shows ✗, your API has a problem even under minimal load.

Seeing threshold failures or connection errors?

  • If you see connection refused errors, confirm your Next.js API is running on localhost:3000. The k6 script makes requests to http://localhost:3000/api/recommendations and will fail if the server is not up.
  • If http_req_failed shows a non-zero rate, check that the /api/recommendations endpoint exists and returns a 200 status. Try opening http://localhost:3000/api/recommendations in your browser first.
  • If you see a Prometheus push error but the test still completes, verify Prometheus is running with the --web.enable-remote-write-receiver flag from Step 1.

help me debug my k6 baseline test errors

Interpret the baseline results

The numbers from your baseline run represent your API's performance under ideal conditions. These are the values you will compare against when adding realistic traffic in the next step.

What do these metrics mean?

  • http_req_duration p(95) = 95% of requests completed within this time. The remaining 5% were slower. This is your primary baseline metric.
  • http_req_duration avg = the mean across all requests. Averages hide outliers, so p(95) is more reliable for setting thresholds.
  • http_req_failed rate = the percentage of requests that returned non-2xx status codes. For a healthy baseline, this should be 0%.
  • iterations = the total number of completed request cycles. With a 20-second hold and 1-second sleep, expect roughly 20 iterations.
  • Write down your baseline values somewhere accessible. In Step 3, you will use these to set thresholds with 30-50% headroom above the observed p(95). For example, if your p(95) is 120ms, you will set a threshold at p(95)<180.
  • Confirm your metrics are visible in Prometheus by opening http://localhost:9090 and searching for any metric with the k6_ prefix.

Why percentiles over averages?

An average of 200ms could mean most requests took 150ms but a few took 2 seconds. The p(95) value gives you a much clearer picture of actual user experience because it captures the slow tail that averages hide.

You now have a documented performance floor for your API. Next up, you will write a realistic multi-scenario load test that simulates actual traffic patterns and sets thresholds based on these baseline numbers.

Write a Realistic Multi-Scenario Load Test

Your baseline test confirmed your API works fine for a single user. But real traffic is never a single user politely waiting between requests.

In this step, you will write a multi-scenario k6 script that simulates two types of users hitting your API at the same time. One group browses recommendations (read-heavy), while another submits preferences (write-heavy). This reveals how your API behaves under realistic concurrent load.

In this step, get ready to:
  • Design a multi-scenario traffic pattern with ramping virtual users.
  • Add response validation checks and a custom latency metric.
  • Run the load test and observe threshold pass/fail results.
Design the traffic pattern

Real applications have different types of users performing different actions simultaneously. Your load test should reflect this by running multiple scenarios in parallel, each with its own traffic shape.

  • Create a new file called load-test.js in the same directory as your baseline.js.
  • Add the imports and custom metric declaration at the top of load-test.js by pasting this code:
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Trend } from 'k6/metrics';

// Custom metric to track recommendation endpoint latency specifically
const recommendationLatency = new Trend('recommendation_latency');

What does this code do?

The first two imports (http, check, sleep) are the same as your baseline test. The new import is Trend from k6/metrics, which lets you create custom metrics that track specific measurements beyond the built-in ones.

The recommendationLatency trend records how long only the recommendations endpoint takes. This gives you a dedicated metric you can set thresholds on and query in Prometheus separately from overall HTTP duration.

  • Below the custom metric, add the options object that defines your two scenarios and thresholds:
export const options = {
  scenarios: {
    // Scenario 1: Users browsing recommendations (read-heavy)
    browse: {
      executor: 'ramping-vus',
      stages: [
        { duration: '15s', target: 20 },
        { duration: '30s', target: 20 },
        { duration: '5s', target: 0 },
      ],
      exec: 'browseRecommendations',
    },
    // Scenario 2: Users submitting preferences (write-heavy, starts 10s later)
    personalize: {
      executor: 'ramping-vus',
      startTime: '10s',
      stages: [
        { duration: '10s', target: 5 },
        { duration: '20s', target: 5 },
        { duration: '5s', target: 0 },
      ],
      exec: 'submitPreferences',
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1000'],
    http_req_failed: ['rate<0.01'],
    checks: ['rate>0.99'],
    recommendation_latency: ['p(95)<600'],
  },
};

What does this code do?

The scenarios object defines two parallel traffic patterns. The browse scenario ramps up to 20 virtual users over 15 seconds, holds steady for 30 seconds, then ramps down. The personalize scenario starts 10 seconds later with 5 virtual users submitting data.

Each scenario uses executor: 'ramping-vus' which smoothly increases and decreases virtual users. The exec property tells k6 which exported function each scenario runs.

The thresholds set automated pass/fail gates. If p95 latency exceeds 500ms, p99 exceeds 1000ms, more than 1% of requests fail, or check pass rate drops below 99%, the test fails. These use headroom above your baseline values.

  • Save load-test.js.

Not sure what thresholds to set?

Start with 30-50% headroom above your baseline values. If your baseline showed p95 of 120ms, a threshold of p(95)<500 gives generous room. You can tighten these after your first successful run.

help me choose threshold values based on my baseline results

Add response validation and custom metrics

Latency thresholds catch slow responses, but they do not catch broken ones. Adding checks on every response ensures your API returns correct data under load, not just fast garbage.

  • Below the options object in load-test.js, add the browseRecommendations function:
export function browseRecommendations() {
  const res = http.get('http://localhost:3000/api/recommendations');

  // Validate response correctness, not just speed
  check(res, {
    'recommendations status 200': (r) => r.status === 200,
    'recommendations has body': (r) => r.body.length > 0,
    'recommendations is JSON': (r) =>
      r.headers['Content-Type'] &&
      r.headers['Content-Type'].includes('application/json'),
  });

  // Record this endpoint's latency in our custom metric
  recommendationLatency.add(res.timings.duration);

  // Random sleep between 1-3 seconds simulates realistic user pacing
  sleep(Math.random() * 2 + 1);
}

What does this code do?

This function runs once per iteration for each virtual user in the browse scenario. It hits the recommendations endpoint, then validates three things: status code is 200, the body is not empty, and the Content-Type header confirms JSON.

The recommendationLatency.add(res.timings.duration) line records this request's duration in your custom Trend metric. This creates a k6_recommendation_latency_p95 time series in Prometheus.

The random sleep (1 to 3 seconds) prevents all virtual users from firing requests in lockstep. Real users pause between actions, and this randomness produces more realistic load patterns.

  • Below browseRecommendations, add the submitPreferences function:
export function submitPreferences() {
  // Generate randomized preference data for each request
  const payload = JSON.stringify({
    temperature: Math.floor(Math.random() * 30) + 5,
    runType: ['easy', 'tempo', 'long', 'interval'][
      Math.floor(Math.random() * 4)
    ],
    terrain: ['road', 'trail', 'track'][Math.floor(Math.random() * 3)],
  });

  const params = {
    headers: { 'Content-Type': 'application/json' },
  };

  const res = http.post(
    'http://localhost:3000/api/preferences',
    payload,
    params
  );

  // Accept both 200 and 201 as valid creation responses
  check(res, {
    'preferences status 200': (r) => r.status === 200 || r.status === 201,
    'preferences is JSON': (r) =>
      r.headers['Content-Type'] &&
      r.headers['Content-Type'].includes('application/json'),
  });

  // Longer pause simulates users filling out a form before submitting
  sleep(Math.random() * 3 + 2);
}

What does this code do?

This function handles the write-heavy scenario. Each virtual user sends a POST request with randomized preference data (temperature, run type, terrain). Randomizing the payload prevents caching from hiding real database performance.

The check accepts both 200 and 201 status codes because APIs may return either for successful creation. The longer sleep (2 to 5 seconds) reflects that users who fill out preference forms take more time between submissions than users who browse.

  • Save load-test.js.
  • Confirm the file exists by running:
ls

You should see both baseline.js and load-test.js in the output.

Seeing a syntax error when you save?

Check that every opening brace { has a matching closing brace }. The most common issue is a missing comma between scenario definitions or between threshold entries.

help me find the syntax error in my load-test.js file

✔️ Awesome, I've got everything!

Great. Double check you have saved load-test.js before moving on.

ⓧ I'd like to double check the full code

import http from 'k6/http';
import { check, sleep } from 'k6';
import { Trend } from 'k6/metrics';

const recommendationLatency = new Trend('recommendation_latency');

export const options = {
  scenarios: {
    browse: {
      executor: 'ramping-vus',
      stages: [
        { duration: '15s', target: 20 },
        { duration: '30s', target: 20 },
        { duration: '5s', target: 0 },
      ],
      exec: 'browseRecommendations',
    },
    personalize: {
      executor: 'ramping-vus',
      startTime: '10s',
      stages: [
        { duration: '10s', target: 5 },
        { duration: '20s', target: 5 },
        { duration: '5s', target: 0 },
      ],
      exec: 'submitPreferences',
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1000'],
    http_req_failed: ['rate<0.01'],
    checks: ['rate>0.99'],
    recommendation_latency: ['p(95)<600'],
  },
};

export function browseRecommendations() {
  const res = http.get('http://localhost:3000/api/recommendations');

  check(res, {
    'recommendations status 200': (r) => r.status === 200,
    'recommendations has body': (r) => r.body.length > 0,
    'recommendations is JSON': (r) =>
      r.headers['Content-Type'] &&
      r.headers['Content-Type'].includes('application/json'),
  });

  recommendationLatency.add(res.timings.duration);

  sleep(Math.random() * 2 + 1);
}

export function submitPreferences() {
  const payload = JSON.stringify({
    temperature: Math.floor(Math.random() * 30) + 5,
    runType: ['easy', 'tempo', 'long', 'interval'][
      Math.floor(Math.random() * 4)
    ],
    terrain: ['road', 'trail', 'track'][Math.floor(Math.random() * 3)],
  });

  const params = {
    headers: { 'Content-Type': 'application/json' },
  };

  const res = http.post(
    'http://localhost:3000/api/preferences',
    payload,
    params
  );

  check(res, {
    'preferences status 200': (r) => r.status === 200 || r.status === 201,
    'preferences is JSON': (r) =>
      r.headers['Content-Type'] &&
      r.headers['Content-Type'].includes('application/json'),
  });

  sleep(Math.random() * 3 + 2);
}
Run the load test and observe thresholds

Your script is ready. Now you will run it with Prometheus output and watch the threshold results in your terminal.

  • Run the full load test with Prometheus remote write by running this command:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw load-test.js

What does this command do?

This is the same Prometheus output pattern you used for the baseline test. The K6_PROMETHEUS_RW_TREND_STATS variable tells k6 which percentile stats to expose as separate Prometheus time series. Setting it to p(95),p(99),avg,min,max creates metrics like k6_http_req_duration_p95 and k6_http_req_duration_p99 in Prometheus.

The test takes about 50 seconds total (the browse scenario runs for 50 seconds, and personalize overlaps starting at the 10 second mark).

When the test finishes, k6 prints a summary with each threshold marked as a pass or fail. Look at the threshold results to see if your API stayed within bounds under concurrent load.

  • In the terminal output, find the thresholds section. Each threshold shows a checkmark or cross next to it.
  • Note any thresholds that are close to breaching (for example, p95 at 480ms with a 500ms limit).
  • Confirm that metrics with the k6_ prefix are streaming into Prometheus by opening http://localhost:9090 and searching for k6_http_req_duration_p95.

Test aborted or all thresholds failed?

If you see many 429 status codes, your Redis rate limiter is kicking in before the API reaches its actual limits. You may need to temporarily increase your rate limit configuration for load testing.

If you see connection refused errors, make sure your Next.js API is still running on localhost:3000.

help me troubleshoot my failing k6 load test thresholds

You now have a repeatable load test that simulates realistic traffic patterns and enforces automated quality gates. Next, you will build a Grafana dashboard to visualize these metrics and identify exactly where your API's bottleneck lives.

Analyze Bottlenecks in Grafana

Your load test is running and streaming metrics into Prometheus. Now you need a way to see those metrics in real time and correlate them with what your application is doing under pressure.

In this step, you will build a Grafana dashboard that visualizes k6 load test data alongside your application metrics. By watching both together during a test run, you can pinpoint exactly which component breaks first as traffic increases.

In this step, get ready to:
  • Build a Grafana dashboard with panels for k6 latency, throughput, error rate, and active virtual users.
  • Re-run the load test while watching the dashboard in real time.
  • Identify and document the specific bottleneck in your system.
Build a k6 load test dashboard in Grafana

A dashboard that shows only HTTP latency tells you something is slow. A dashboard that correlates latency with database query time, CPU usage, and connection counts tells you why it is slow.

  • Switch to your Grafana instance in the browser (http://localhost:3000 by default, or wherever your Grafana is accessible).
  • Click Dashboards in the left-hand main menu.
  • Click New and select New Dashboard.
  • Click the Add new element icon (the blue plus sign) and drag a panel onto the dashboard.
  • On the new panel, select Configure visualization.
  • In the Queries tab, confirm that Prometheus is selected as your data source.
  • Switch to Code mode in the query editor and enter the following PromQL query:
k6_http_req_duration_p95
  • Title the panel p95 Latency and click Save.
  • Repeat the process to add four more panels with the following PromQL queries and titles:
  • k6_http_req_duration_p99 titled p99 Latency.
  • k6_http_reqs titled Throughput (RPS).
  • k6_http_req_failed titled Error Rate.
  • k6_vus titled Active VUs.
  • Save the dashboard with a title like k6 Load Test Dashboard.

Why correlate with application panels?

If your existing Grafana setup already has panels for CPU usage, memory, PostgreSQL query duration, or Redis hit/miss rates, add them to this same dashboard (or open them side-by-side). When latency spikes, you want to immediately see whether the database, the cache, or the CPU is responsible.

The k6 panels show you what is happening from the client perspective. Your application panels show you where the problem lives inside the system.

  • Verify your dashboard by checking that each panel shows data. If you ran a load test recently (within the last 15 minutes), you should see time-series lines on the latency and throughput panels.

Panels showing "No data"?

If panels are empty, the k6_ metrics may have expired from Prometheus. Set the time range in the top-right corner to cover when you last ran a test. If you still see nothing, re-run the pipeline test from Step 1 to push fresh metrics.

Also confirm your Prometheus data source is selected (not the default Grafana data source).

Help me troubleshoot empty k6 panels in Grafana

Identify the bottleneck

A bottleneck is the component in your system that degrades first as load increases. Watching the dashboard in real time while traffic ramps up reveals exactly where the system starts to struggle.

  • Re-run your load test while watching the Grafana dashboard by running this command:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw load-test.js
  • Switch to your Grafana dashboard immediately after running the command.
  • Set the time range to Last 5 minutes and the refresh interval to 5s (top-right corner of Grafana) so the panels update in near real time.

What to look for during the test

Watch for these signals as virtual users ramp up:

  • Latency inflection point: at what VU count does p95 suddenly spike upward?
  • Error rate onset: do errors (the k6_http_req_failed panel) start climbing at a specific load level?
  • Database saturation: does your PostgreSQL query duration panel spike before HTTP latency does? If so, the database is your bottleneck.
  • Rate limiter behavior: do you see 429 responses from your Redis rate limiting? If so, the limiter is triggering before the true application ceiling.

The bottleneck is whichever component degrades first as load increases.

  • Once the test completes (~50 seconds), note the VU count and time at which p95 latency spiked.
  • Compare the spike timing with your application panels (database query duration, CPU, memory) to identify which component degraded first.

Dashboard not updating during the test?

k6 pushes metrics to Prometheus every 5 seconds by default. If your dashboard stays flat, confirm the refresh interval is set to 5s and that the time range covers the current moment.

If latency panels stay at zero, verify the K6_PROMETHEUS_RW_TREND_STATS variable includes p(95),p(99) in your run command.

Help me debug my Grafana dashboard not updating during k6 tests

Document the finding

A bottleneck finding is only useful if it is specific and measurable. Vague statements like "the API is slow" give you nothing to act on. Precise documentation gives you a clear target for Step 5.

  • Record which component is the bottleneck (e.g., PostgreSQL, Redis, CPU).
  • Record at what load level the degradation appears (e.g., 15+ concurrent VUs).
  • Record the metric evidence that proves it (e.g., query duration doubling from 50ms to 100ms, causing p95 HTTP latency to exceed 400ms).
  • Note which endpoint is the slowest. In most cases for this project, it will be the /api/recommendations endpoint because it queries the database for personalized results.

Example bottleneck statement

A well-documented bottleneck looks like: "PostgreSQL query duration doubles at 15+ concurrent users, causing p95 HTTP latency to exceed 400ms. The recommendations endpoint is the slowest."

This gives you three things: the component (PostgreSQL), the trigger (15+ VUs), and the proof (query duration + HTTP p95 metric). In the next step, you will target this specific component with an optimization.

You now have concrete evidence of where your system struggles under load. Next up, you will apply a targeted optimization to that bottleneck and re-run the test to prove the improvement with real numbers.

Optimize and Verify Improvement

In Step 4, you pinpointed the exact component dragging your API down under load. Now it's time to fix it and prove the fix works with hard numbers.

Performance engineering is only credible when you can show measurable before-and-after improvement. In this step, you will apply one targeted optimization, re-run your load test, and compare the results in Grafana.

In this step, get ready to:
  • Apply a targeted optimization based on your bottleneck analysis.
  • Re-run the load test with a tagged identifier for before/after comparison.
  • Document measurable improvement in Grafana with visual proof.
Apply a targeted optimization

Before you change anything, you need a clean tagged reference run. This gives Grafana a distinct testid label to filter on when comparing results later.

  • Create a tagged "before" reference by running your load test with the before-optimization tag:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw --tag testid=before-optimization load-test.js

Why tag the run now?

The --tag testid=before-optimization flag attaches a label to every metric this test emits. In Prometheus, you can filter by testid to isolate this specific run from all others.

Your earlier untagged runs are still in Prometheus, but having a cleanly tagged pair makes the Grafana comparison straightforward.

  • Wait for the test to complete and confirm it finishes without errors in the terminal output.

Now apply your optimization. Based on the bottleneck you identified in Step 4, pick ONE of the following fixes that targets your specific problem.

  • Database index (if PostgreSQL query duration was your bottleneck): Add an index on the column used in your slowest query. Update your Prisma schema with an @@index directive, then apply it by running:
  • Connection pooling (if you saw connection exhaustion under load): Increase the pool size in your database URL by appending ?connection_limit=10 to the connection string in your .env file.
  • Redis caching (if the same expensive read endpoint was called repeatedly): Add Redis caching for the recommendations endpoint. Check the cache before querying the database.
  • Query optimization (if you are returning full records when only a few fields are needed): Use Prisma select to return only the fields your API response actually uses.
npx prisma migrate dev

When do I run the migration?

Only run npx prisma migrate dev if your optimization involves a schema change (like adding a database index). Connection pooling, caching, and query optimizations are code-level changes that take effect when you restart your Next.js dev server.

  • After applying your optimization, restart your Next.js development server so the changes take effect.

Pick the smallest change

The goal is a single, isolated fix so you can attribute any improvement directly to that change. Applying multiple optimizations at once makes it impossible to know which one helped.

Not sure which optimization to pick?

Return to your Step 4 findings. If database query duration spiked first, start with an index or connection pool increase. If the same endpoint was slow but the database looked fine, try Prisma select to reduce payload processing time.

Help me choose the right optimization based on my bottleneck findings.

Re-run the load test and compare

With your optimization in place and the dev server restarted, it is time to run the exact same load test against the improved API. Using the same script ensures any difference in results comes from your optimization, not a change in test conditions.

  • Run the load test with the after-optimization tag by running this command:
K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write \
K6_PROMETHEUS_RW_TREND_STATS=p(95),p(99),avg,min,max \
k6 run -o experimental-prometheus-rw --tag testid=after-optimization load-test.js

What does this command do?

This is the same command you used for the "before" run, with one difference: --tag testid=after-optimization labels all metrics from this run with a distinct identifier.

Prometheus now holds two labeled datasets: before-optimization and after-optimization. Grafana can filter on testid to overlay both runs on the same panel.

  • When the test finishes, note the key metrics from the terminal summary:
  • Compare these values to the pre-optimization run you noted earlier.

Numbers look the same as before?

Make sure your Next.js dev server was restarted after applying the optimization. Code-level changes like caching or select optimization only take effect after a server restart.

If you added a database index, confirm the migration applied successfully by checking for errors in the npx prisma migrate dev output.

Help me figure out why my optimization did not improve performance.

Document the improvement in Grafana

Terminal numbers prove the improvement exists. A Grafana visualization makes it undeniable by showing both runs as overlaid time series.

  • Switch back to your Grafana dashboard in the browser.
  • On your k6_http_req_duration_p95 panel, click the panel title and select Edit.
  • In the query editor, add a label filter for testid. Create two queries: one filtered to testid="before-optimization" and one filtered to testid="after-optimization".
  • You should see two distinct lines on the same panel showing latency over time for each run.

What should I see?

The after-optimization line should sit visibly below the before-optimization line on latency panels. On throughput panels, the after line may sit higher (more requests per second handled).

Even a small visible gap between the lines represents a real, repeatable improvement you can point to as evidence.

  • Repeat the same comparison on your k6_http_reqs panel (throughput) and k6_http_req_failed panel (error rate) to see the full picture.
  • Record your findings. Note the before and after values for p95 latency, p99 latency, throughput (requests per second), and error rate.

Save the dashboard

Click the save icon (or press Cmd+S on macOS, Ctrl+S on Windows) to persist your dashboard with the comparison queries. This becomes your documented proof of improvement.

Secret mission

Find Your Breaking Point

You know your API performs well under expected load. But what happens when traffic keeps climbing? In this mission, you'll push your API to its absolute limit to discover exactly how many concurrent users it can handle before things fall apart.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally with open-source tools, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • k6 binary installed via Homebrew.
  • Prometheus remote write receiver (--web.enable-remote-write-receiver flag added to your Prometheus startup).
  • Grafana dashboard panels for k6 load test metrics.
  • Test script files: baseline.js, load-test.js, and breaking.js.

Keep everything running

No action needed. Choose this if you plan to continue performance testing your API or want k6 available for future projects.

  • k6 remains installed and ready for future load tests at no cost.
  • The Prometheus remote write receiver stays enabled with no performance or cost impact.
  • Your Grafana dashboard panels persist for future test comparisons.
  • Your test scripts stay on disk for re-running or adapting to other projects.

Pause - I'll come back to this later

There are no persistent processes or running resources to pause in this project. Everything either runs on demand (k6 tests) or is configuration-only (Prometheus flag, Grafana panels).

  • Your k6 scripts, Grafana dashboard, and Prometheus configuration all stay in place without consuming resources.
  • When you return, you can re-run your tests immediately without any setup.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to fully clean up.

  • Uninstall k6 by running:
brew uninstall k6
  • Remove the --web.enable-remote-write-receiver flag from your Prometheus startup command (in your Docker Compose file or docker run command). Restart Prometheus for the change to take effect.
  • Delete your k6 Grafana dashboard. Open the dashboard in Grafana, click the gear icon to open dashboard settings, then select the option to delete the dashboard.
  • Delete the test script files by running:
rm baseline.js load-test.js breaking.js
  • Confirm the files are removed by running:
ls *.js

You should see no results, or only files unrelated to this project.

Nice Work!

Nice Work!

You just built a complete performance engineering workflow for your Next.js API. From baseline measurement to bottleneck identification to verified optimization, you now have repeatable proof that your system handles realistic traffic within defined SLOs.

You've learned how to:

  • Install k6 and connect it to an existing Prometheus/Grafana observability stack via remote write, streaming load test metrics in real time.
  • Write multi-scenario load tests with p95/p99 thresholds, custom trend metrics, and automated pass/fail gates that catch regressions before users do.
  • Identify performance bottlenecks through Grafana metric correlation and prove measurable improvement by re-running the same test against your optimized API.
  • Secret Mission: Discover your system's exact breaking point and capacity ceiling using progressive VU ramping with abortOnFail thresholds.

Ready to quiz yourself?