Build an LLM Prompt Evaluation Dashboard

Build a full-stack tool to test and score LLM prompts with a React dashboard.

Introduction

30 Second Summary

Every time you tweak a prompt in an AI application, you are guessing whether it improved or got worse. Companies building with LLMs need a structured way to test prompt changes before shipping them to real users.

In this project, you will build a full-stack prompt evaluation dashboard that runs test cases against the Gemini API and scores the results using both deterministic checks and AI-powered judging.

What You'll Build

A full-stack TypeScript developer tool with an Express backend and React frontend for defining prompt test suites, running them against an LLM, and visualizing evaluation results.

By the end of this project, you'll have:

  • An Express API backend with endpoints for managing eval suites and triggering evaluation runs.
  • An eval runner engine with five pluggable scorers: exact match, contains, regex, JSON schema, and LLM-as-judge.
  • A React dashboard deployed to Vercel that displays pass/fail badges, latency stats, and side-by-side output comparisons.
  • Secret Mission: Add prompt versioning with A/B comparison to test two prompt variants side-by-side and see which performs better.

Are there any prerequisites?

This project assumes familiarity with Node.js, React, and TypeScript. You also need an active Google account to obtain a free Gemini API key from Google AI Studio.

Set Up the Project

Your evaluation dashboard is a full-stack application with two separate pieces: a backend API that runs prompts through Gemini and scores the outputs, and a frontend dashboard that visualizes the results.

In this step, you'll scaffold a monorepo with separate server and client directories, configure TypeScript for the server, wire up an Express skeleton, and verify your Gemini API connection works.

In this step, get ready to:
  • Scaffold the monorepo with server and client directories.
  • Configure TypeScript and create the Express server skeleton.
  • Store your Gemini API key and verify the connection.
Scaffold the monorepo structure

A monorepo keeps your backend and frontend in one repository while treating them as independent apps. This makes deployment and local development straightforward.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and the server source directory by running these commands:
mkdir prompt-eval-dashboard
cd prompt-eval-dashboard
mkdir -p server/src
  • Confirm the folders exist by running:
ls

You should see a server directory in the output.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code (or Cursor if that's your editor) and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop and select the prompt-eval-dashboard folder, then click Open.

Now create the server's package.json file. This defines your server's dependencies and run scripts.

  • In the Explorer sidebar, right-click the server folder and select New File.
  • Name the file package.json and paste the following code:
{
  "name": "eval-server",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "build": "tsc",
    "start": "node dist/index.js"
  },
  "dependencies": {
    "@google/genai": "^2.8.0",
    "cors": "^2.8.5",
    "dotenv": "^16.4.7",
    "express": "^4.19.2"
  },
  "devDependencies": {
    "@types/cors": "^2.8.17",
    "@types/express": "^5.0.6",
    "tsx": "^4.19.0",
    "typescript": "^5.7.0"
  }
}

What does this file do?

The "type": "module" setting enables modern ES Module import/export syntax instead of the older require() style.

The dev script uses tsx to run TypeScript directly without a separate compile step. The watch flag auto-restarts when you save changes.

  • Save server/package.json.
  • Open the integrated terminal by pressing Ctrl+` (backtick key, same shortcut on macOS and Windows).
  • Install all the server dependencies by running these commands:
cd server
npm install

You should see a node_modules folder appear in the Explorer sidebar under server/.

Seeing permission errors or network failures?

Make sure you're connected to the internet. If you see EACCES permission errors on macOS/Linux, try prefixing the command with sudo. On Windows, open your terminal as Administrator.

help me fix npm install errors in my prompt-eval-dashboard project

  • Navigate back to the project root to scaffold the React frontend:
cd ..
npm create vite@latest client -- --template react-ts

What will Vite ask me?

Vite may show interactive prompts even though you specified the template. If asked about rolldown-vite (Experimental), select No. If asked about ESLint, select either option (it won't affect this project).

If asked whether to install dependencies and start the dev server immediately, select No. You'll install client dependencies later.

  • Verify both directories exist by running:
ls

You should see both server and client directories listed.

Configure TypeScript and create the Express server

TypeScript needs a configuration file to understand how to compile your code. The server uses ESM (ES Modules) with modern compilation settings.

  • In the Explorer sidebar, right-click the server folder and select New File.
  • Name the file tsconfig.json and paste the following configuration:
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "bundler",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "include": ["src"]
}

What do these settings do?

  • "module": "ESNext" tells TypeScript to output modern import/export syntax.
  • "moduleResolution": "bundler" allows imports without file extensions, matching how bundlers resolve modules.
  • "strict": true enables all strict type-checking flags for safer code.
  • Save server/tsconfig.json.

Now create the Express app skeleton that will serve your API routes.

  • In the Explorer sidebar, expand server then right-click the src folder and select New File.
  • Name the file index.ts and paste the following code:
import 'dotenv/config';
import express from 'express';
import cors from 'cors';

// Create the Express application
const app = express();

// Enable CORS so the React frontend can call this API
app.use(cors());

// Parse incoming JSON request bodies
app.use(express.json());

// Start the server on port 3001 (or whatever PORT is set in the environment)
const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
  console.log(`Eval server running on port ${PORT}`);
});

What does this code do?

  • import 'dotenv/config' loads environment variables from your .env file into process.env automatically.
  • cors() allows the React frontend (running on a different port) to make requests to this server without browser security blocks.
  • express.json() parses JSON request bodies so your route handlers can read req.body directly.
  • Save server/src/index.ts.
  • In the integrated terminal, make sure you're in the server directory and start the dev server by running:
cd server
npm run dev

You should see Eval server running on port 3001 in the terminal output.

Not seeing the server start?

Make sure your terminal is inside the server directory. Run pwd to check your current location. If you're in the project root, run cd server first.

help me debug why my Express server won't start

  • Stop the server by pressing Ctrl+C in the terminal.

✔️ Awesome, I've got everything!

Great. Double check you've saved all your files before moving on.

ⓧ I'd like to double check the full code

Here is the complete server/src/index.ts at this point:

import 'dotenv/config';
import express from 'express';
import cors from 'cors';

const app = express();
app.use(cors());
app.use(express.json());

const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
  console.log(`Eval server running on port ${PORT}`);
});

And the complete server/tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "bundler",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "include": ["src"]
}
Store your API key and verify the connection

Your server needs the Gemini API key to call the model. Environment variables keep secrets out of your source code so they're never committed to version control.

  • Right-click the server folder in the Explorer sidebar and select New File.
  • Name the file .env and paste the following, replacing your_api_key_here with the actual Gemini API key you obtained from Google AI Studio:
GEMINI_API_KEY=your_api_key_here

Why a .env file?

The .env file keeps your API key separate from your code. The dotenv package loads it into process.env at runtime. Never commit this file to version control.

  • Save server/.env.

Now verify that your API key works by running a quick test call to the Gemini API.

  • Create a temporary test file by right-clicking the src folder inside server and selecting New File.
  • Name the file test-connection.ts and paste the following code:
import 'dotenv/config';
import { GoogleGenAI } from '@google/genai';

// Initialize the Gemini client with your API key
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY! });

// Make a simple test call to verify the connection
const response = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'Say hello in one sentence.',
});

console.log('Gemini response:', response.text);

What does this test do?

This sends a simple prompt to Gemini 2.5 Flash and prints the response. If your API key is valid, you'll see a short greeting in the terminal. The Gemini free tier covers this test at no cost.

  • Save server/src/test-connection.ts.
  • In the integrated terminal, make sure you're in the server directory and run the test:
npx tsx src/test-connection.ts

You should see Gemini response: followed by a short greeting from the model.

Seeing an API key error or empty response?

Double check that your .env file is saved inside the server folder (not the project root). Make sure there are no extra spaces around the = sign in GEMINI_API_KEY=your_key.

If you see a 429 rate limit error, wait a minute and try again. The Gemini free tier has per-minute rate limits.

help me fix my Gemini API connection

  • Once you see a successful response, delete the test file by running:
rm src/test-connection.ts

The test file was only for verification. Your actual LLM calls will go through a dedicated runner in the next step.

Your project skeleton is ready and your Gemini connection is confirmed. Next up, you'll build the evaluation engine that runs prompts through the model and scores the outputs.

Build the Eval Engine

Your server skeleton is running. Now you need the brain behind the evaluation tool.

An eval engine takes a prompt, sends it to an LLM, then scores the response using a mix of rule-based checks and AI-powered judging. In this step, you will build the full scoring pipeline from TypeScript interfaces all the way to a Gemini-powered judge.

In this step, get ready to:
  • Define TypeScript interfaces for eval suites, test cases, and results.
  • Build four deterministic scorers that test LLM outputs without calling an API.
  • Create the LLM runner and an AI-powered judge scorer using Gemini.
Define the eval types and deterministic scorers

Before writing any logic, you need a clear contract for what an eval test case looks like, what a result contains, and how scorers report their verdicts. TypeScript interfaces give you that contract.

  • Create a new file called types.ts inside your server/src folder.
  • Add the following type definitions:
export interface EvalCase {
  id: string;
  input: string;
  expectedOutput: string;
  scorers: ScorerType[];
}

export type ScorerType = "exactMatch" | "contains" | "regex" | "jsonSchema" | "llmJudge";

export interface EvalSuite {
  id: string;
  name: string;
  systemPrompt: string;
  cases: EvalCase[];
}

export interface EvalCaseResult {
  caseId: string;
  input: string;
  expectedOutput: string;
  actualOutput: string;
  scores: Record<ScorerType, number>;
  passed: boolean;
  latencyMs: number;
}

export interface EvalRunResult {
  suiteId: string;
  suiteName: string;
  timestamp: string;
  results: EvalCaseResult[];
  aggregatePassRate: number;
  averageLatencyMs: number;
}

What do these types represent?

  • EvalCase is one test. It has an input prompt, an expected output to compare against, and a list of scorers to apply.
  • ScorerType is a union of the five scoring strategies your engine supports.
  • EvalSuite groups multiple test cases under one system prompt.
  • EvalCaseResult captures everything about a single test run: what the LLM actually said, each scorer's verdict, and how long it took.
  • EvalRunResult is the top-level summary with aggregate stats.
  • Save types.ts.

✔️ Awesome, I've got everything!

Great. Make sure you have saved types.ts before moving on.

ⓧ I'd like to double check the full code

export interface EvalCase {
  id: string;
  input: string;
  expectedOutput: string;
  scorers: ScorerType[];
}

export type ScorerType = "exactMatch" | "contains" | "regex" | "jsonSchema" | "llmJudge";

export interface EvalSuite {
  id: string;
  name: string;
  systemPrompt: string;
  cases: EvalCase[];
}

export interface EvalCaseResult {
  caseId: string;
  input: string;
  expectedOutput: string;
  actualOutput: string;
  scores: Record<ScorerType, number>;
  passed: boolean;
  latencyMs: number;
}

export interface EvalRunResult {
  suiteId: string;
  suiteName: string;
  timestamp: string;
  results: EvalCaseResult[];
  aggregatePassRate: number;
  averageLatencyMs: number;
}

Now you need functions that compare LLM outputs against expected values without making any API calls. These are your deterministic scorers.

  • Create a new file called scorers.ts inside server/src.
  • Add the four scorer functions and a dispatcher:
import { ScorerType } from "./types.js";

export function scoreExactMatch(expected: string, actual: string): number {
  return expected.trim() === actual.trim() ? 1 : 0;
}

export function scoreContains(expected: string, actual: string): number {
  return actual.toLowerCase().includes(expected.toLowerCase()) ? 1 : 0;
}

export function scoreRegex(pattern: string, actual: string): number {
  try {
    const regex = new RegExp(pattern, "i");
    return regex.test(actual) ? 1 : 0;
  } catch {
    return 0;
  }
}

export function scoreJsonSchema(expected: string, actual: string): number {
  try {
    JSON.parse(actual);
    return 1;
  } catch {
    return 0;
  }
}

export function getDeterministicScore(
  scorer: ScorerType,
  expected: string,
  actual: string
): number | null {
  switch (scorer) {
    case "exactMatch":
      return scoreExactMatch(expected, actual);
    case "contains":
      return scoreContains(expected, actual);
    case "regex":
      return scoreRegex(expected, actual);
    case "jsonSchema":
      return scoreJsonSchema(expected, actual);
    default:
      return null;
  }
}

What does each scorer check?

  • scoreExactMatch returns 1 only if the trimmed output matches the expected value exactly.
  • scoreContains returns 1 if the expected string appears anywhere in the output (case-insensitive).
  • scoreRegex tests the output against a regex pattern. If the pattern is invalid, it returns 0 instead of crashing.
  • scoreJsonSchema checks whether the output is valid JSON. This is useful for prompts that ask for structured responses.
  • getDeterministicScore is the dispatcher. It routes each scorer type to the correct function and returns null for the llmJudge type, which needs an API call instead.

Why deterministic scorers?

Deterministic scorers run instantly, cost nothing, and produce repeatable results. They handle the cases where you know exactly what a good response looks like. The LLM judge (coming next) handles fuzzier evaluations where meaning matters more than exact wording.

  • Save scorers.ts.

Seeing a red underline on the import?

If your editor shows an error on import { ScorerType } from "./types.js", make sure you included the .js extension. TypeScript with ESM modules requires file extensions in import paths, even though the source file is .ts.

help me fix this TypeScript import error with ESM modules

✔️ Awesome, I've got everything!

Great. Make sure you have saved scorers.ts before moving on.

ⓧ I'd like to double check the full code

import { ScorerType } from "./types.js";

export function scoreExactMatch(expected: string, actual: string): number {
  return expected.trim() === actual.trim() ? 1 : 0;
}

export function scoreContains(expected: string, actual: string): number {
  return actual.toLowerCase().includes(expected.toLowerCase()) ? 1 : 0;
}

export function scoreRegex(pattern: string, actual: string): number {
  try {
    const regex = new RegExp(pattern, "i");
    return regex.test(actual) ? 1 : 0;
  } catch {
    return 0;
  }
}

export function scoreJsonSchema(expected: string, actual: string): number {
  try {
    JSON.parse(actual);
    return 1;
  } catch {
    return 0;
  }
}

export function getDeterministicScore(
  scorer: ScorerType,
  expected: string,
  actual: string
): number | null {
  switch (scorer) {
    case "exactMatch":
      return scoreExactMatch(expected, actual);
    case "contains":
      return scoreContains(expected, actual);
    case "regex":
      return scoreRegex(expected, actual);
    case "jsonSchema":
      return scoreJsonSchema(expected, actual);
    default:
      return null;
  }
}

Your types and scorers are ready. Now set up a data store with sample test cases so you can verify the engine works end-to-end later.

  • Create a new file called store.ts inside server/src.
  • Add the in-memory storage and a seeded demo suite:
import { EvalSuite, EvalRunResult } from "./types.js";

export const suites: Map<string, EvalSuite> = new Map();
export const runs: EvalRunResult[] = [];

const demoSuite: EvalSuite = {
  id: "demo-1",
  name: "Summarization Quality",
  systemPrompt: "You are a concise summarizer. Summarize the given text in one sentence.",
  cases: [
    {
      id: "case-1",
      input: "The quick brown fox jumps over the lazy dog. This sentence contains every letter of the alphabet and is commonly used for typing practice.",
      expectedOutput: "pangram",
      scorers: ["contains", "llmJudge"],
    },
    {
      id: "case-2",
      input: "JavaScript was created in 1995 by Brendan Eich while working at Netscape. It has since become the most widely used programming language for web development.",
      expectedOutput: "JavaScript",
      scorers: ["contains", "llmJudge"],
    },
    {
      id: "case-3",
      input: "Return the answer as JSON with a 'summary' field.",
      expectedOutput: '{"summary": ""}',
      scorers: ["jsonSchema"],
    },
  ],
};

suites.set(demoSuite.id, demoSuite);

What does this store do?

This file acts as a simple in-memory database. The suites Map holds eval suite definitions keyed by ID. The runs array stores historical run results.

The demo suite tests a summarization prompt with three cases. Case 1 checks that the summary mentions "pangram". Case 2 checks for "JavaScript". Case 3 checks that the output is valid JSON. This gives you something to test against immediately.

  • Save store.ts.

✔️ Awesome, I've got everything!

Great. Make sure you have saved store.ts before moving on.

ⓧ I'd like to double check the full code

import { EvalSuite, EvalRunResult } from "./types.js";

export const suites: Map<string, EvalSuite> = new Map();
export const runs: EvalRunResult[] = [];

const demoSuite: EvalSuite = {
  id: "demo-1",
  name: "Summarization Quality",
  systemPrompt: "You are a concise summarizer. Summarize the given text in one sentence.",
  cases: [
    {
      id: "case-1",
      input: "The quick brown fox jumps over the lazy dog. This sentence contains every letter of the alphabet and is commonly used for typing practice.",
      expectedOutput: "pangram",
      scorers: ["contains", "llmJudge"],
    },
    {
      id: "case-2",
      input: "JavaScript was created in 1995 by Brendan Eich while working at Netscape. It has since become the most widely used programming language for web development.",
      expectedOutput: "JavaScript",
      scorers: ["contains", "llmJudge"],
    },
    {
      id: "case-3",
      input: "Return the answer as JSON with a 'summary' field.",
      expectedOutput: '{"summary": ""}',
      scorers: ["jsonSchema"],
    },
  ],
};

suites.set(demoSuite.id, demoSuite);
Implement the LLM runner

Deterministic scorers handle cases where you know the exact answer. But many prompts produce open-ended text where meaning matters more than exact wording. For those, you need to call an LLM and measure what comes back.

  • Create a new file called llm.ts inside server/src.
  • Add the @google/genai client setup and the runPrompt function:
import { GoogleGenAI } from "@google/genai";

// Initialize the Gemini client using your API key from .env
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY! });

export async function runPrompt(
  systemPrompt: string,
  userInput: string
): Promise<{ text: string; latencyMs: number }> {
  // Record the start time so we can measure latency
  const start = Date.now();
  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: userInput,
    config: {
      systemInstruction: systemPrompt,
      temperature: 0.1,
    },
  });
  const latencyMs = Date.now() - start;
  return { text: response.text ?? "", latencyMs };
}

What does runPrompt do?

  • GoogleGenAI creates a client instance that reads your GEMINI_API_KEY from the environment.
  • systemInstruction sets the persona for the model (e.g., "You are a concise summarizer"). This comes from the eval suite definition.
  • temperature: 0.1 makes outputs nearly deterministic. Lower temperature means less randomness, which is important for repeatable evaluations.
  • The function returns both the response text and how long the call took. Latency tracking lets you spot slow prompts in the dashboard later.
  • Save llm.ts.

Red underline on GoogleGenAI?

Make sure you installed the @google/genai package in the previous step. If the import is unresolved, run npm install again inside your server folder.

help me fix the @google/genai import error

Add the LLM-as-judge scorer

The contains scorer can tell you whether the word "pangram" appeared. But it cannot tell you whether the summary was coherent or relevant. For that, you ask another LLM to judge the quality.

  • In llm.ts, add the llmJudge function below the runPrompt function:
export async function llmJudge(
  input: string,
  expected: string,
  actual: string
): Promise<number> {
  // Build a prompt that asks Gemini to score the response quality
  const judgePrompt = `You are an evaluation judge. Score the following LLM response on a scale of 0 to 1 based on how well it matches the expected output in meaning and quality.

Input prompt: "${input}"
Expected output: "${expected}"
Actual output: "${actual}"

Respond with ONLY a number between 0 and 1. Nothing else.`;

  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: judgePrompt,
    config: { temperature: 0 },
  });
  // Parse the numeric score and clamp it between 0 and 1
  const score = parseFloat(response.text ?? "0");
  return isNaN(score) ? 0 : Math.min(1, Math.max(0, score));
}

What does the LLM judge do?

The judge sends a meta-prompt to Gemini that includes the original input, what the expected response should look like, and what the model actually said. It asks for a single number between 0 and 1.

Temperature is set to 0 for maximum consistency. The Math.min(1, Math.max(0, score)) clamp ensures the score stays in range even if the model returns something unexpected.

  • Save llm.ts.
  • Return to your terminal and start the dev server by running:
npm run dev

You should see Eval server running on port 3001 in your terminal with no TypeScript errors. This confirms all four files compile correctly together.

Seeing a TypeScript compilation error?

  • Check that every import path ends with .js (e.g., "./types.js"). ESM module resolution requires explicit extensions.
  • Make sure GEMINI_API_KEY is defined in your .env file. The ! after process.env.GEMINI_API_KEY tells TypeScript you guarantee the value exists.
  • Confirm that your tsconfig.json has "module": "ESNext" and "moduleResolution": "bundler".

help me debug this TypeScript compilation error in my eval server

✔️ Awesome, I've got everything!

Great. Double check you have saved llm.ts with both the runPrompt and llmJudge functions.

ⓧ I'd like to double check the full code

import { GoogleGenAI } from "@google/genai";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY! });

export async function runPrompt(
  systemPrompt: string,
  userInput: string
): Promise<{ text: string; latencyMs: number }> {
  const start = Date.now();
  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: userInput,
    config: {
      systemInstruction: systemPrompt,
      temperature: 0.1,
    },
  });
  const latencyMs = Date.now() - start;
  return { text: response.text ?? "", latencyMs };
}

export async function llmJudge(
  input: string,
  expected: string,
  actual: string
): Promise<number> {
  const judgePrompt = `You are an evaluation judge. Score the following LLM response on a scale of 0 to 1 based on how well it matches the expected output in meaning and quality.

Input prompt: "${input}"
Expected output: "${expected}"
Actual output: "${actual}"

Respond with ONLY a number between 0 and 1. Nothing else.`;

  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: judgePrompt,
    config: { temperature: 0 },
  });
  const score = parseFloat(response.text ?? "0");
  return isNaN(score) ? 0 : Math.min(1, Math.max(0, score));
}

Your eval engine is fully built. You have four instant scorers for rule-based checks and an AI-powered judge for semantic evaluation. Next up, you will wire these into Express API endpoints so the React dashboard can trigger evaluation runs.

Build the API Layer

In Step 2, you built the eval engine's core pieces: deterministic scorers, an LLM runner, and an LLM-as-judge function. Now these modules need an entry point that your React frontend can communicate with.

In this step, you'll create REST API endpoints that tie everything together. The Express server will expose routes to list eval suites, create new ones, and execute evaluation runs that orchestrate the scorers and LLM calls.

In this step, get ready to:
  • Wire up Express routes for suite management.
  • Implement the evaluation run endpoint that orchestrates scorers and LLM calls.
  • Verify the API works end-to-end with the seeded demo suite.
Create the suite management routes

Your Express server currently has middleware configured but no routes. You need to import the modules you built in Step 2 and expose them through HTTP endpoints.

  • In server/src/index.ts, find the existing import lines at the top of the file:
import 'dotenv/config';
import express from 'express';
import cors from 'cors';
  • Replace those three import lines with this expanded import block:
import "dotenv/config";
import express from "express";
import cors from "cors";
import { suites, runs } from "./store.js";
import { runPrompt, llmJudge } from "./llm.js";
import { getDeterministicScore } from "./scorers.js";
import { EvalSuite, EvalCaseResult, EvalRunResult, ScorerType } from "./types.js";

What do these imports do?

Each import pulls in a module you created in Step 2. suites and runs give the routes access to in-memory storage.

runPrompt and llmJudge handle Gemini API calls. getDeterministicScore applies the pattern-matching scorers. The type imports enforce TypeScript safety across the request/response boundary.

  • Now find the const PORT line near the bottom of the file.
  • Add the following two route handlers above that line, after app.use(express.json());:
app.get("/api/suites", (_req, res) => {
  res.json(Array.from(suites.values()));
});

app.post("/api/suites", (req, res) => {
  const suite: EvalSuite = { ...req.body, id: `suite-${Date.now()}` };
  suites.set(suite.id, suite);
  res.status(201).json(suite);
});

What do these routes do?

GET /api/suites converts the suites Map into an array and returns it as JSON. The frontend will call this to populate the sidebar.

POST /api/suites accepts a new suite definition from the request body, generates a unique ID using Date.now(), stores it in the Map, and returns the created suite with a 201 status.

  • Save server/src/index.ts.

Seeing red squiggles in your editor?

If TypeScript reports errors on the import paths, make sure each path ends with .js (e.g. ./store.js). ESM modules in Node.js require file extensions in import paths even for TypeScript files.

help me fix TypeScript import errors in my Express server

Implement the evaluation run endpoint

The run endpoint is where the real work happens. When called, it iterates through every test case in a suite, sends each prompt to Gemini, scores the response, and returns structured results.

  • In server/src/index.ts, add the following route handler directly below the app.post("/api/suites", ...) route you just created:
app.post("/api/suites/:id/run", async (req, res) => {
  const suite = suites.get(req.params.id);
  if (!suite) {
    res.status(404).json({ error: "Suite not found" });
    return;
  }

  const results: EvalCaseResult[] = [];

  for (const evalCase of suite.cases) {
    // Run the prompt through Gemini
    const { text, latencyMs } = await runPrompt(suite.systemPrompt, evalCase.input);
    const scores: Record<string, number> = {};

    // Apply each scorer to the response
    for (const scorer of evalCase.scorers) {
      const deterministicScore = getDeterministicScore(scorer, evalCase.expectedOutput, text);
      if (deterministicScore !== null) {
        scores[scorer] = deterministicScore;
      } else if (scorer === "llmJudge") {
        scores[scorer] = await llmJudge(evalCase.input, evalCase.expectedOutput, text);
      }
    }

    // Calculate pass/fail based on average score
    const avgScore = Object.values(scores).reduce((a, b) => a + b, 0) / Object.values(scores).length;

    results.push({
      caseId: evalCase.id,
      input: evalCase.input,
      expectedOutput: evalCase.expectedOutput,
      actualOutput: text,
      scores: scores as Record<ScorerType, number>,
      passed: avgScore >= 0.5,
      latencyMs,
    });
  }

  // Build aggregate stats for the full run
  const runResult: EvalRunResult = {
    suiteId: suite.id,
    suiteName: suite.name,
    timestamp: new Date().toISOString(),
    results,
    aggregatePassRate: results.filter((r) => r.passed).length / results.length,
    averageLatencyMs: results.reduce((a, r) => a + r.latencyMs, 0) / results.length,
  };

  runs.push(runResult);
  res.json(runResult);
});

What does this endpoint do?

  • It looks up the suite by ID from the URL parameter. If not found, it returns a 404.
  • For each test case, it calls runPrompt to get the actual LLM response and measures latency.
  • Each scorer assigned to the case runs against the response. Deterministic scorers return instantly. The llmJudge scorer makes a second Gemini call for semantic evaluation.
  • A test case passes if its average score across all scorers is at least 0.5.
  • The endpoint returns per-case results plus aggregate stats (overall pass rate and average latency).
  • Save server/src/index.ts.

TypeScript complaining about async handler?

Make sure the route callback has the async keyword before (req, res). Without it, the await calls inside the handler will cause a compile error.

help me fix the async route handler error

Test the API with the demo suite

The in-memory store you created in Step 2 already has a demo eval suite seeded with three test cases. You can start the server and hit the endpoints to confirm everything is wired up.

  • Start the dev server by running this command in your terminal (from the server directory):
npm run dev
  • You should see Eval server running on port 3001 in the terminal output.
  • Open a second terminal and test the GET endpoint by running:
curl http://localhost:3001/api/suites
  • You should see a JSON array containing the demo suite with the name Summarization Quality and three test cases.

What about the run endpoint?

You can test POST /api/suites/demo-1/run with curl -X POST http://localhost:3001/api/suites/demo-1/run, but it will make real calls to the Gemini API. Each test case takes a few seconds to process.

If you see results with scores and pass/fail statuses, your eval engine is fully functional. If you see a 429 error, you may have hit a rate limit. The deterministic scorers still work without LLM calls.

Server won't start?

Check that your .env file exists in the server directory with your GEMINI_API_KEY set. Also confirm you're running npm run dev from inside the server folder, not the project root.

help me debug why my Express server won't start

✔️ Awesome, I've got everything!

Great. Double check you've saved server/src/index.ts and your server is returning the demo suite JSON.

ⓧ I'd like to double check the full code

import "dotenv/config";
import express from "express";
import cors from "cors";
import { suites, runs } from "./store.js";
import { runPrompt, llmJudge } from "./llm.js";
import { getDeterministicScore } from "./scorers.js";
import { EvalSuite, EvalCaseResult, EvalRunResult, ScorerType } from "./types.js";

const app = express();
app.use(cors());
app.use(express.json());

app.get("/api/suites", (_req, res) => {
  res.json(Array.from(suites.values()));
});

app.post("/api/suites", (req, res) => {
  const suite: EvalSuite = { ...req.body, id: `suite-${Date.now()}` };
  suites.set(suite.id, suite);
  res.status(201).json(suite);
});

app.post("/api/suites/:id/run", async (req, res) => {
  const suite = suites.get(req.params.id);
  if (!suite) {
    res.status(404).json({ error: "Suite not found" });
    return;
  }

  const results: EvalCaseResult[] = [];

  for (const evalCase of suite.cases) {
    const { text, latencyMs } = await runPrompt(suite.systemPrompt, evalCase.input);
    const scores: Record<string, number> = {};

    for (const scorer of evalCase.scorers) {
      const deterministicScore = getDeterministicScore(scorer, evalCase.expectedOutput, text);
      if (deterministicScore !== null) {
        scores[scorer] = deterministicScore;
      } else if (scorer === "llmJudge") {
        scores[scorer] = await llmJudge(evalCase.input, evalCase.expectedOutput, text);
      }
    }

    const avgScore = Object.values(scores).reduce((a, b) => a + b, 0) / Object.values(scores).length;

    results.push({
      caseId: evalCase.id,
      input: evalCase.input,
      expectedOutput: evalCase.expectedOutput,
      actualOutput: text,
      scores: scores as Record<ScorerType, number>,
      passed: avgScore >= 0.5,
      latencyMs,
    });
  }

  const runResult: EvalRunResult = {
    suiteId: suite.id,
    suiteName: suite.name,
    timestamp: new Date().toISOString(),
    results,
    aggregatePassRate: results.filter((r) => r.passed).length / results.length,
    averageLatencyMs: results.reduce((a, r) => a + r.latencyMs, 0) / results.length,
  };

  runs.push(runResult);
  res.json(runResult);
});

const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
  console.log(`Eval server running on port ${PORT}`);
});

Your API layer is live and serving evaluation data. Next up, you'll build the React dashboard that calls these endpoints and visualizes the results.

Build the React Dashboard

Your eval engine and API are ready to score prompts. Now you need a way to see the results without squinting at raw JSON in the terminal.

In this step, you will build a React dashboard that fetches eval suites from your API, runs evaluations on demand, and displays pass/fail results with latency metrics and side-by-side output comparisons.

In this step, get ready to:
  • Set up the dashboard layout with a sidebar and main content area.
  • Build the evaluation results display with pass/fail badges, score tables, and expandable detail rows.
  • Style the dashboard with a dark theme and responsive layout.
Set up the dashboard layout and component logic

The Vite scaffold generated a default App.tsx with a counter demo. You will replace it entirely with your evaluation dashboard component.

  • Open client/src/App.tsx in your editor.
  • Delete all existing content and replace it with the following dashboard component:
import { useState, useEffect } from "react";
import "./App.css";

interface EvalSuite {
  id: string;
  name: string;
  systemPrompt: string;
  cases: { id: string; input: string; expectedOutput: string; scorers: string[] }[];
}

interface EvalCaseResult {
  caseId: string;
  input: string;
  expectedOutput: string;
  actualOutput: string;
  scores: Record<string, number>;
  passed: boolean;
  latencyMs: number;
}

interface EvalRunResult {
  suiteId: string;
  suiteName: string;
  timestamp: string;
  results: EvalCaseResult[];
  aggregatePassRate: number;
  averageLatencyMs: number;
}

const API_URL = import.meta.env.VITE_API_URL || "http://localhost:3001";

function App() {
  const [suites, setSuites] = useState<EvalSuite[]>([]);
  const [selectedSuite, setSelectedSuite] = useState<EvalSuite | null>(null);
  const [runResult, setRunResult] = useState<EvalRunResult | null>(null);
  const [loading, setLoading] = useState(false);
  const [expandedCase, setExpandedCase] = useState<string | null>(null);

  useEffect(() => {
    fetch(`${API_URL}/api/suites`)
      .then((r) => r.json())
      .then(setSuites);
  }, []);

  async function runEval(suiteId: string) {
    setLoading(true);
    setRunResult(null);
    const res = await fetch(`${API_URL}/api/suites/${suiteId}/run`, { method: "POST" });
    const data = await res.json();
    setRunResult(data);
    setLoading(false);
  }

  return (
    <div className="app">
      <aside className="sidebar">
        <h2>Eval Suites</h2>
        <ul>
          {suites.map((s) => (
            <li
              key={s.id}
              className={selectedSuite?.id === s.id ? "active" : ""}
              onClick={() => { setSelectedSuite(s); setRunResult(null); }}
            >
              {s.name}
            </li>
          ))}
        </ul>
      </aside>

      <main className="content">
        {!selectedSuite && <p className="placeholder">Select an eval suite to get started.</p>}

        {selectedSuite && (
          <>
            <header className="suite-header">
              <h1>{selectedSuite.name}</h1>
              <p className="system-prompt">{selectedSuite.systemPrompt}</p>
              <button onClick={() => runEval(selectedSuite.id)} disabled={loading}>
                {loading ? "Running..." : "Run Evaluation"}
              </button>
            </header>

            {runResult && (
              <section className="results">
                <div className="stats">
                  <div className={`pass-rate ${runResult.aggregatePassRate >= 0.7 ? "good" : "bad"}`}>
                    {Math.round(runResult.aggregatePassRate * 100)}% Pass
                  </div>
                  <div className="latency">
                    Avg Latency: {Math.round(runResult.averageLatencyMs)}ms
                  </div>
                </div>

                <table className="results-table">
                  <thead>
                    <tr>
                      <th>Status</th>
                      <th>Input</th>
                      <th>Scores</th>
                      <th>Latency</th>
                    </tr>
                  </thead>
                  <tbody>
                    {runResult.results.map((r) => (
                      <>
                        <tr
                          key={r.caseId}
                          className={r.passed ? "passed" : "failed"}
                          onClick={() => setExpandedCase(expandedCase === r.caseId ? null : r.caseId)}
                        >
                          <td>{r.passed ? "PASS" : "FAIL"}</td>
                          <td className="input-cell">{r.input.slice(0, 60)}...</td>
                          <td>
                            {Object.entries(r.scores).map(([k, v]) => (
                              <span key={k} className="score-badge">
                                {k}: {v.toFixed(2)}
                              </span>
                            ))}
                          </td>
                          <td>{r.latencyMs}ms</td>
                        </tr>
                        {expandedCase === r.caseId && (
                          <tr key={`${r.caseId}-detail`} className="detail-row">
                            <td colSpan={4}>
                              <div className="comparison">
                                <div>
                                  <h4>Expected</h4>
                                  <pre>{r.expectedOutput}</pre>
                                </div>
                                <div>
                                  <h4>Actual</h4>
                                  <pre>{r.actualOutput}</pre>
                                </div>
                              </div>
                            </td>
                          </tr>
                        )}
                      </>
                    ))}
                  </tbody>
                </table>
              </section>
            )}
          </>
        )}
      </main>
    </div>
  );
}

export default App;

What does this component do?

This single component handles the entire dashboard. Here are the key sections:

  • Interfaces (EvalSuite, EvalCaseResult, EvalRunResult) define the shape of data coming from your API. These mirror the server types.
  • API_URL reads from a VITE_API_URL environment variable, falling back to http://localhost:3001 for local development.
  • useEffect fetches all eval suites on mount so the sidebar populates immediately.
  • runEval POSTs to your /api/suites/:id/run endpoint and stores the result in state.
  • The sidebar lists suites with an active class for highlighting. The main area shows results after a run completes.
  • Save client/src/App.tsx.

Seeing TypeScript errors in your editor?

  • Make sure you deleted ALL the default Vite starter code before pasting. Leftover imports or the old component will conflict.
  • Check that the client folder has its node_modules installed. If not, run npm install inside the client directory.

Help me fix TypeScript errors in my App.tsx file.

Understand the results display logic

The results section of your dashboard does the heavy lifting of turning raw eval data into something readable. Let's trace through how it works.

How does the results display work?

The results only render when runResult is not null (after the learner clicks **Run Evaluation**). Three things display:

  • A pass-rate badge that shows the aggregate percentage. It gets a good class (green) when 70% or more pass, or a bad class (red) otherwise.
  • A results table with one row per test case. Each row shows PASS/FAIL status, a truncated input, individual scorer badges, and latency in milliseconds.
  • Expandable detail rows that toggle when you click a table row. The expandedCase state tracks which row is open. The detail row shows expected vs actual output in a two-column grid.

Why keep everything in one component?

For a tool this size, a single App.tsx keeps the data flow obvious. State lives in one place, props don't need threading, and the entire render tree is visible at a glance.

If you were to scale this into a production app, you would extract the sidebar, results table, and comparison view into separate component files. For a portfolio demo, clarity beats architecture.

Style the dashboard with dark-theme CSS

The component references CSS classes like .sidebar, .pass-rate, and .results-table that do not exist yet. You need to replace the default Vite CSS with your own styles.

  • Open client/src/App.css and replace all existing content with the following styles:
* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
  background: #0f1117;
  color: #e1e4e8;
}

.app {
  display: flex;
  min-height: 100vh;
}

.sidebar {
  width: 260px;
  background: #161b22;
  padding: 24px 16px;
  border-right: 1px solid #30363d;
}

.sidebar h2 {
  font-size: 14px;
  text-transform: uppercase;
  letter-spacing: 0.5px;
  color: #8b949e;
  margin-bottom: 16px;
}

.sidebar ul {
  list-style: none;
}

.sidebar li {
  padding: 10px 12px;
  border-radius: 6px;
  cursor: pointer;
  margin-bottom: 4px;
  font-size: 14px;
}

.sidebar li:hover {
  background: #21262d;
}

.sidebar li.active {
  background: #1f6feb;
  color: white;
}

.content {
  flex: 1;
  padding: 32px;
}

.placeholder {
  color: #8b949e;
  font-size: 16px;
  margin-top: 100px;
  text-align: center;
}

.suite-header {
  margin-bottom: 32px;
}

.suite-header h1 {
  font-size: 24px;
  margin-bottom: 8px;
}

.system-prompt {
  color: #8b949e;
  font-size: 14px;
  margin-bottom: 16px;
  font-style: italic;
}

button {
  background: #238636;
  color: white;
  border: none;
  padding: 10px 20px;
  border-radius: 6px;
  font-size: 14px;
  cursor: pointer;
}

button:hover {
  background: #2ea043;
}

button:disabled {
  background: #21262d;
  color: #8b949e;
  cursor: not-allowed;
}

.stats {
  display: flex;
  gap: 16px;
  margin-bottom: 24px;
}

.pass-rate {
  font-size: 24px;
  font-weight: bold;
  padding: 12px 20px;
  border-radius: 8px;
}

.pass-rate.good {
  background: #0d1117;
  border: 1px solid #238636;
  color: #3fb950;
}

.pass-rate.bad {
  background: #0d1117;
  border: 1px solid #da3633;
  color: #f85149;
}

.latency {
  display: flex;
  align-items: center;
  font-size: 14px;
  color: #8b949e;
}

.results-table {
  width: 100%;
  border-collapse: collapse;
}

.results-table th {
  text-align: left;
  padding: 12px;
  border-bottom: 1px solid #30363d;
  color: #8b949e;
  font-size: 12px;
  text-transform: uppercase;
}

.results-table td {
  padding: 12px;
  border-bottom: 1px solid #21262d;
  font-size: 14px;
}

.results-table tr {
  cursor: pointer;
}

.results-table tr:hover {
  background: #161b22;
}

tr.passed td:first-child {
  color: #3fb950;
  font-weight: bold;
}

tr.failed td:first-child {
  color: #f85149;
  font-weight: bold;
}

.input-cell {
  max-width: 300px;
  overflow: hidden;
  text-overflow: ellipsis;
  white-space: nowrap;
}

.score-badge {
  display: inline-block;
  background: #21262d;
  padding: 2px 8px;
  border-radius: 4px;
  margin-right: 4px;
  font-size: 12px;
}

.detail-row td {
  background: #161b22;
  padding: 20px;
}

.comparison {
  display: grid;
  grid-template-columns: 1fr 1fr;
  gap: 16px;
}

.comparison h4 {
  margin-bottom: 8px;
  color: #8b949e;
  font-size: 12px;
  text-transform: uppercase;
}

.comparison pre {
  background: #0d1117;
  padding: 12px;
  border-radius: 6px;
  font-size: 13px;
  white-space: pre-wrap;
  word-break: break-word;
}

What do these styles do?

  • The dark background (#0f1117) with light text (#e1e4e8) creates a developer-tool aesthetic similar to GitHub's dark mode.
  • The .pass-rate.good and .pass-rate.bad classes use green and red borders to give instant visual feedback on evaluation health.
  • The .comparison grid splits the expanded detail row into two columns so the learner can compare expected and actual output side by side.
  • Save client/src/App.css.
  • Also delete client/src/index.css if it exists, or remove the import './index.css' line from client/src/main.tsx so the Vite default styles do not conflict with yours.
  • Start the client dev server by running this command from the client folder:
npm run dev
  • Make sure your Express server is still running in a separate terminal (from the server folder, npm run dev).
  • Open the URL Vite prints in the terminal (typically http://localhost:5173) in your browser.
  • You should see a dark-themed page with a sidebar listing "Summarization Quality" on the left.
  • Click **Summarization Quality** in the sidebar. The suite name, system prompt, and a green **Run Evaluation** button appear in the main area.
  • Click **Run Evaluation**. The button changes to "Running..." while the API processes the test cases. After a few seconds, you should see a pass-rate badge, latency stat, and a table of results.
  • Click any table row to expand it and see the expected vs actual output side by side.

Sidebar is empty or Run Evaluation fails?

  • Open your browser's developer console (F12). If you see a CORS or network error, your Express server is not running on port 3001. Switch to the server terminal and run npm run dev.
  • If the page loads but the sidebar is blank, check that your server's GET /api/suites returns JSON by visiting http://localhost:3001/api/suites directly in your browser.
  • If Run Evaluation returns an error about the Gemini API, check that your .env file contains a valid GEMINI_API_KEY value.

Help me debug why my React dashboard can't connect to the Express API.

✔️ Awesome, I've got everything!

Great. Double check you have saved both App.tsx and App.css.

ⓧ I'd like to double check the full code

Here is the full client/src/App.tsx:

import { useState, useEffect } from "react";
import "./App.css";

interface EvalSuite {
  id: string;
  name: string;
  systemPrompt: string;
  cases: { id: string; input: string; expectedOutput: string; scorers: string[] }[];
}

interface EvalCaseResult {
  caseId: string;
  input: string;
  expectedOutput: string;
  actualOutput: string;
  scores: Record<string, number>;
  passed: boolean;
  latencyMs: number;
}

interface EvalRunResult {
  suiteId: string;
  suiteName: string;
  timestamp: string;
  results: EvalCaseResult[];
  aggregatePassRate: number;
  averageLatencyMs: number;
}

const API_URL = import.meta.env.VITE_API_URL || "http://localhost:3001";

function App() {
  const [suites, setSuites] = useState<EvalSuite[]>([]);
  const [selectedSuite, setSelectedSuite] = useState<EvalSuite | null>(null);
  const [runResult, setRunResult] = useState<EvalRunResult | null>(null);
  const [loading, setLoading] = useState(false);
  const [expandedCase, setExpandedCase] = useState<string | null>(null);

  useEffect(() => {
    fetch(`${API_URL}/api/suites`)
      .then((r) => r.json())
      .then(setSuites);
  }, []);

  async function runEval(suiteId: string) {
    setLoading(true);
    setRunResult(null);
    const res = await fetch(`${API_URL}/api/suites/${suiteId}/run`, { method: "POST" });
    const data = await res.json();
    setRunResult(data);
    setLoading(false);
  }

  return (
    <div className="app">
      <aside className="sidebar">
        <h2>Eval Suites</h2>
        <ul>
          {suites.map((s) => (
            <li
              key={s.id}
              className={selectedSuite?.id === s.id ? "active" : ""}
              onClick={() => { setSelectedSuite(s); setRunResult(null); }}
            >
              {s.name}
            </li>
          ))}
        </ul>
      </aside>

      <main className="content">
        {!selectedSuite && <p className="placeholder">Select an eval suite to get started.</p>}

        {selectedSuite && (
          <>
            <header className="suite-header">
              <h1>{selectedSuite.name}</h1>
              <p className="system-prompt">{selectedSuite.systemPrompt}</p>
              <button onClick={() => runEval(selectedSuite.id)} disabled={loading}>
                {loading ? "Running..." : "Run Evaluation"}
              </button>
            </header>

            {runResult && (
              <section className="results">
                <div className="stats">
                  <div className={`pass-rate ${runResult.aggregatePassRate >= 0.7 ? "good" : "bad"}`}>
                    {Math.round(runResult.aggregatePassRate * 100)}% Pass
                  </div>
                  <div className="latency">
                    Avg Latency: {Math.round(runResult.averageLatencyMs)}ms
                  </div>
                </div>

                <table className="results-table">
                  <thead>
                    <tr>
                      <th>Status</th>
                      <th>Input</th>
                      <th>Scores</th>
                      <th>Latency</th>
                    </tr>
                  </thead>
                  <tbody>
                    {runResult.results.map((r) => (
                      <>
                        <tr
                          key={r.caseId}
                          className={r.passed ? "passed" : "failed"}
                          onClick={() => setExpandedCase(expandedCase === r.caseId ? null : r.caseId)}
                        >
                          <td>{r.passed ? "PASS" : "FAIL"}</td>
                          <td className="input-cell">{r.input.slice(0, 60)}...</td>
                          <td>
                            {Object.entries(r.scores).map(([k, v]) => (
                              <span key={k} className="score-badge">
                                {k}: {v.toFixed(2)}
                              </span>
                            ))}
                          </td>
                          <td>{r.latencyMs}ms</td>
                        </tr>
                        {expandedCase === r.caseId && (
                          <tr key={`${r.caseId}-detail`} className="detail-row">
                            <td colSpan={4}>
                              <div className="comparison">
                                <div>
                                  <h4>Expected</h4>
                                  <pre>{r.expectedOutput}</pre>
                                </div>
                                <div>
                                  <h4>Actual</h4>
                                  <pre>{r.actualOutput}</pre>
                                </div>
                              </div>
                            </td>
                          </tr>
                        )}
                      </>
                    ))}
                  </tbody>
                </table>
              </section>
            )}
          </>
        )}
      </main>
    </div>
  );
}

export default App;

And the full client/src/App.css:

* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
  background: #0f1117;
  color: #e1e4e8;
}

.app {
  display: flex;
  min-height: 100vh;
}

.sidebar {
  width: 260px;
  background: #161b22;
  padding: 24px 16px;
  border-right: 1px solid #30363d;
}

.sidebar h2 {
  font-size: 14px;
  text-transform: uppercase;
  letter-spacing: 0.5px;
  color: #8b949e;
  margin-bottom: 16px;
}

.sidebar ul {
  list-style: none;
}

.sidebar li {
  padding: 10px 12px;
  border-radius: 6px;
  cursor: pointer;
  margin-bottom: 4px;
  font-size: 14px;
}

.sidebar li:hover {
  background: #21262d;
}

.sidebar li.active {
  background: #1f6feb;
  color: white;
}

.content {
  flex: 1;
  padding: 32px;
}

.placeholder {
  color: #8b949e;
  font-size: 16px;
  margin-top: 100px;
  text-align: center;
}

.suite-header {
  margin-bottom: 32px;
}

.suite-header h1 {
  font-size: 24px;
  margin-bottom: 8px;
}

.system-prompt {
  color: #8b949e;
  font-size: 14px;
  margin-bottom: 16px;
  font-style: italic;
}

button {
  background: #238636;
  color: white;
  border: none;
  padding: 10px 20px;
  border-radius: 6px;
  font-size: 14px;
  cursor: pointer;
}

button:hover {
  background: #2ea043;
}

button:disabled {
  background: #21262d;
  color: #8b949e;
  cursor: not-allowed;
}

.stats {
  display: flex;
  gap: 16px;
  margin-bottom: 24px;
}

.pass-rate {
  font-size: 24px;
  font-weight: bold;
  padding: 12px 20px;
  border-radius: 8px;
}

.pass-rate.good {
  background: #0d1117;
  border: 1px solid #238636;
  color: #3fb950;
}

.pass-rate.bad {
  background: #0d1117;
  border: 1px solid #da3633;
  color: #f85149;
}

.latency {
  display: flex;
  align-items: center;
  font-size: 14px;
  color: #8b949e;
}

.results-table {
  width: 100%;
  border-collapse: collapse;
}

.results-table th {
  text-align: left;
  padding: 12px;
  border-bottom: 1px solid #30363d;
  color: #8b949e;
  font-size: 12px;
  text-transform: uppercase;
}

.results-table td {
  padding: 12px;
  border-bottom: 1px solid #21262d;
  font-size: 14px;
}

.results-table tr {
  cursor: pointer;
}

.results-table tr:hover {
  background: #161b22;
}

tr.passed td:first-child {
  color: #3fb950;
  font-weight: bold;
}

tr.failed td:first-child {
  color: #f85149;
  font-weight: bold;
}

.input-cell {
  max-width: 300px;
  overflow: hidden;
  text-overflow: ellipsis;
  white-space: nowrap;
}

.score-badge {
  display: inline-block;
  background: #21262d;
  padding: 2px 8px;
  border-radius: 4px;
  margin-right: 4px;
  font-size: 12px;
}

.detail-row td {
  background: #161b22;
  padding: 20px;
}

.comparison {
  display: grid;
  grid-template-columns: 1fr 1fr;
  gap: 16px;
}

.comparison h4 {
  margin-bottom: 8px;
  color: #8b949e;
  font-size: 12px;
  text-transform: uppercase;
}

.comparison pre {
  background: #0d1117;
  padding: 12px;
  border-radius: 6px;
  font-size: 13px;
  white-space: pre-wrap;
  word-break: break-word;
}

Your dashboard is live and visualizing real eval results. Next up, you will deploy the entire project to Vercel so it becomes a shareable live demo.

Deploy to Vercel

You have a working eval dashboard running locally. The final piece is getting it live on the internet so you can share it in recruiter outreach and link it on your portfolio.

In this step, you will configure Vercel to serve both the React frontend and Express API from a single deployment, push your code to GitHub, and deploy to a live URL.

In this step, get ready to:
  • Configure a vercel.json that routes API and frontend requests correctly.
  • Push your project to GitHub and deploy via Vercel.
  • Verify the live deployment works end-to-end.
Configure the project for Vercel deployment

Vercel needs to know how to handle two types of requests: API calls that should go to your Express server, and everything else that should serve the React frontend.

A vercel.json file at the project root tells Vercel how to route incoming traffic using rewrites.

  • Create a vercel.json file in the root of your project (the same level as your server/ and client/ folders) and paste the following configuration:
{
  "rewrites": [
    { "source": "/api/(.*)", "destination": "/server/dist/index.js" },
    { "source": "/(.*)", "destination": "/client/dist/index.html" }
  ]
}

What does this configuration do?

The rewrites array defines two routing rules that Vercel evaluates in order.

  • The first rule matches any request starting with /api/ and sends it to your compiled Express server at /server/dist/index.js.
  • The second rule catches all other requests and serves the React app's index.html from the built client folder.
  • Save vercel.json.

Why rewrites instead of redirects?

Rewrites forward requests internally without changing the URL in the browser. Your dashboard calls /api/suites and that path stays visible in the address bar, even though Vercel routes it to the Express function behind the scenes.

Push to GitHub and deploy on Vercel

Vercel deploys automatically from a connected GitHub repository. You need to push your code to GitHub first, then import the repo into Vercel.

  • Create a .gitignore file in the project root with the following content to exclude dependencies, environment variables, and build output:
node_modules
.env
dist
  • Initialize a Git repository and commit your code by running these commands:
git init
git add -A
git commit -m "feat: complete eval dashboard"
  • Go to github.com and create a new repository.
  • Name it llm-eval-dashboard (or your preferred name).
  • Do not initialize with a README, .gitignore, or license (you already have code).
  • Copy the remote URL GitHub gives you.
  • Push your local code to GitHub by running these commands:
git remote add origin [[YOUR_REPO_URL="https://github.com/YOUR_USERNAME/llm-eval-dashboard.git"]]
git branch -M main
git push -u origin main
  • Go to vercel.com and sign in.
  • Click the Add New... button, then select Project.
  • In the Import Git Repository section, find your llm-eval-dashboard repository and click Import.

Don't see your repository?

If this is your first time connecting GitHub to Vercel, you may need to authorize the Vercel GitHub integration. Follow the prompts to grant access to your repositories.

Before clicking Deploy, you need to add your API key as an environment variable so the server can access it in production.

  • On the project configuration page, expand the Environment Variables section.
  • In the Name field, enter GEMINI_API_KEY.
  • In the Value field, paste your Gemini API key (the same value from your local .env file).
  • Click Deploy.

Vercel will build your project and provide a production URL once the deployment finishes.

Build failing?

  • Check that vercel.json is in the project root (not inside server/ or client/).
  • Confirm your .gitignore does not exclude vercel.json or your source files.
  • Verify the GEMINI_API_KEY environment variable is set in the Vercel dashboard.

Help me debug my Vercel deployment build failure.

Verify the live deployment

A successful deployment means nothing if the app does not work end-to-end. You will now confirm that both the frontend and backend respond correctly on the live URL.

  • Open your production URL (shown in the Vercel dashboard after deployment).
  • You should see the eval dashboard with Summarization Quality listed in the sidebar.
  • Click Summarization Quality in the sidebar to select it.
  • Click the Run Evaluation button.
  • Wait for the evaluation to complete (the button shows "Running..." while in progress).
  • Confirm that results appear: a pass rate badge, latency stats, and a table of test case results with PASS or FAIL status.

What should I see?

The dashboard should display a percentage pass rate (green if 70% or above, red otherwise), an average latency in milliseconds, and a results table. Clicking a row expands it to show the expected vs actual output side-by-side.

Dashboard loads but Run Evaluation fails?

  • Open the browser console (F12) and check for network errors on the /api/suites/:id/run request.
  • Go back to your Vercel project dashboard, select Settings in the sidebar, then Environment Variables. Confirm GEMINI_API_KEY is listed with the correct value.
  • If you just added or changed the variable, you need to redeploy for it to take effect.

Help me debug why my eval run fails on the deployed Vercel app.

✔️ Awesome, I've got everything!

Great. Make sure you have saved all files and that your deployment is showing results correctly.

ⓧ I'd like to double check the full code

Here is the vercel.json file that should be in your project root:

{
  "rewrites": [
    { "source": "/api/(.*)", "destination": "/server/dist/index.js" },
    { "source": "/(.*)", "destination": "/client/dist/index.html" }
  ]
}

Your eval dashboard is live on the internet. You can share this URL in recruiter messages, embed it in your portfolio, and point to it as proof that you understand AI developer tooling from the infrastructure layer up.

Secret mission

Prompt Versioning with A/B Comparison

Your eval dashboard runs prompts and scores them. But in real AI teams, the question is never just 'does this prompt work?' It's 'does this prompt work better than the last version?' Add A/B comparison so you can test two prompt variants side-by-side and declare a winner.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on the Gemini API free tier and Vercel's free hosting plan. There are no ongoing costs to worry about.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Vercel project (hosting your frontend and API).
  • GitHub repository (source code).
  • Gemini API key (stored in Vercel environment variables and your local .env file).
  • Local project files (the folder containing your server and client directories on your Desktop).

Keep everything running

No action needed. Choose this if you are still actively building or want the live demo available for recruiter outreach.

  • Your Vercel deployment stays live on the free tier indefinitely at no cost.
  • Your GitHub repo remains accessible for sharing.
  • Your Gemini API key remains active on the free tier with no billing attached.
  • Your local project files stay on your Desktop.

Pause - I'll come back to this later

Disable LLM calls while keeping the frontend live. This is useful if you want to preserve the deployment but stop any API usage.

  • Open your project on the Vercel dashboard.
  • Click Settings in the sidebar, then select Environment Variables.
  • Delete the GEMINI_API_KEY variable. This disables all LLM calls while your frontend remains accessible.
  • When you are ready to resume, re-add the GEMINI_API_KEY environment variable with your key value and redeploy.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below.

  • Delete the Vercel project: Open the project on vercel.com. Click Settings in the sidebar. At the bottom of the General page, find the Delete Project section. Click Delete, enter your project name to confirm, then click Continue. This also removes your deployed environment variables.
  • Delete the GitHub repository: Open your repo on GitHub. Click Settings. Scroll down to the Danger Zone section and click Delete this repository. Confirm by typing the repository name, then click the confirmation button.
  • Delete the Gemini API key: Open the API Keys page in Google AI Studio. Locate the key you created for this project and delete it.
  • Delete local project files: Remove the project folder from your Desktop by running the command below for your operating system.

macOS

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your-project-folder-name"]]

Windows

cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_FOLDER="your-project-folder-name"]]

Nice Work!

Nice Work!

You just built a full-stack LLM prompt evaluation dashboard from scratch using Express, React, and TypeScript. This is the kind of internal tooling that AI companies build to ship better prompts faster.

You've learned how to:

  • Build a full-stack LLM evaluation engine with Express and TypeScript that runs prompts against the Gemini API and scores outputs.
  • Implement 4 deterministic scorers (exact match, contains, regex, JSON schema) plus an LLM-as-judge scorer for semantic quality evaluation.
  • Create a React TypeScript dashboard with real-time eval visualization, pass/fail badges, latency tracking, and side-by-side output comparisons.
  • Deploy a live demo to Vercel that you can link in recruiter outreach and portfolio sites.
  • Secret Mission: Added prompt A/B comparison with side-by-side version testing and an aggregate winner badge.

Ready to quiz yourself?