Build a TypeScript Code Review CLI

Build a CLI tool that simulates the RLHF code review workflow in TypeScript.

Introduction

30 Second Summary

Every time an AI assistant writes better code, it's because a human engineer reviewed two options and explained which was better. This compare-rank-explain loop is how AI models learn to produce higher quality output.

In this project, you will build a TypeScript CLI tool that simulates the code review workflow used in RLHF training roles. You'll compare code snippet pairs, rank them, write technical justifications, and practice the exact skills these roles test for.

What You'll Build

A command-line code review tool called ts-reviewer that presents pairs of TypeScript code snippets with intentional issues and prompts you to rank them, explain your reasoning, and produce a refactored version.

By the end of this project, you'll have:

  • A working TypeScript CLI application that runs interactive code review challenges.
  • A bank of code snippet pairs covering type safety issues, performance traps, security vulnerabilities, and logic errors.
  • A structured JSON output recording your rankings and written justifications for each review.
  • Secret Mission: Add a scoring rubric that evaluates your reviews against expected issues and shows what you missed.

Are there any prerequisites?

This project assumes familiarity with TypeScript syntax and basic Node.js usage. You don't need any prior RLHF or AI training experience.

Set Up Your TypeScript Project

Before you can build the review engine, you need a properly configured TypeScript project. TypeScript's strict mode catches bugs at compile time, which is exactly the kind of rigor an RLHF reviewer needs to demonstrate.

In this step, you'll create the project directory, install TypeScript and tsx (the tool that runs your TypeScript files directly), and configure everything for modern module resolution.

In this step, get ready to:
  • Initialize a TypeScript project with npm and install dependencies.
  • Configure tsconfig.json with strict mode and modern module settings.
  • Verify the setup by running a test file with tsx.
Initialize the project and install dependencies

Every Node.js project starts with a package.json file that tracks your project's name, scripts, and dependencies.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ts-reviewer
cd ts-reviewer
  • Confirm the folder was created by running:
ls ~/Desktop

You should see ts-reviewer listed in the output.

  • Initialize the project by running:
npm init -y

What does this command do?

The -y flag tells npm to skip all the interactive prompts and create a package.json with default values. You'll replace these defaults in a moment.

  • Install TypeScript and tsx as dev dependencies by running:
npm install -D typescript tsx

You should see output confirming packages were added, with no errors.

What is tsx?

tsx stands for TypeScript Execute. It enhances Node.js with esbuild to run TypeScript files directly, without a separate compile step.

This means you can run npx tsx src/index.ts and get instant feedback. No build folder, no waiting.

Seeing permission errors?

If you see EACCES errors, your npm permissions need fixing. Try prefixing the command with sudo (macOS/Linux) or running your terminal as Administrator (Windows).

Help me fix npm permission errors

Now you need to update package.json with the correct project settings and scripts. The default file from npm init is missing the module type and review scripts your CLI needs.

  • Open package.json in Vim by running:
vim package.json
  • Select all existing content and delete it by typing ggdG in normal mode.
  • Enter insert mode by pressing i.
  • Paste the following content into the file:
{
  "name": "ts-reviewer",
  "version": "1.0.0",
  "description": "TypeScript code review CLI for RLHF training",
  "type": "module",
  "scripts": {
    "start": "npx tsx src/index.ts",
    "review": "npx tsx src/index.ts",
    "review:easy": "npx tsx src/index.ts easy",
    "review:medium": "npx tsx src/index.ts medium",
    "review:hard": "npx tsx src/index.ts hard"
  },
  "devDependencies": {
    "typescript": "^6.0.3",
    "tsx": "^4.22.3"
  }
}

What does this configuration do?

  • "type": "module" tells Node.js to treat .js files as ES modules, enabling modern import/export syntax.
  • The scripts section defines shortcuts for running your CLI at different difficulty levels.
  • Each script uses npx tsx to run TypeScript directly without a compile step.
  • Save and exit Vim by pressing Esc, then typing :wq and pressing Enter.
  • Verify the file saved correctly by running:
cat package.json

You should see the JSON content you just pasted, including "type": "module" and your five scripts.

File looks garbled or empty?

If the output looks wrong, re-open the file with vim package.json and try again. Make sure you press i before pasting so Vim is in insert mode.

Help me fix my package.json in Vim

Configure TypeScript for the project

The tsconfig.json file controls how TypeScript checks your code. Strict mode enables every type-checking rule, catching the same kinds of issues you'll be reviewing in the RLHF challenges.

  • Create and open tsconfig.json in Vim by running:
vim tsconfig.json
  • Enter insert mode by pressing i.
  • Paste the following configuration:
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "outDir": "./dist",
    "rootDir": "./src"
  },
  "include": ["src/**/*"]
}

What do these settings do?

  • "strict": true enables all strict type-checking options. This catches bugs like implicit any types and null reference errors at compile time.
  • "module": "NodeNext" and "moduleResolution": "NodeNext" tell TypeScript to use Node.js's native ES module resolution. Import paths use .js extensions.
  • "target": "ES2022" compiles to modern JavaScript features like top-level await and Array.at().
  • "rootDir": "./src" tells TypeScript where your source files live. "outDir": "./dist" is where compiled JavaScript would go if you ran the compiler.
  • Save and exit Vim by pressing Esc, then typing :wq and pressing Enter.
  • Verify the file saved correctly by running:
cat tsconfig.json

You should see your configuration with "strict": true and "module": "NodeNext" displayed.

Seeing unexpected characters?

If the output has extra characters or broken JSON, delete the file with rm tsconfig.json and recreate it. Make sure Vim is in insert mode (press i) before you paste.

Help me fix my tsconfig.json

Create the file structure and verify the setup

Your TypeScript source files will live in the src/ directory. You need to create this folder and confirm that tsx can execute TypeScript files from it.

  • Create the source directory by running:
mkdir src
  • Confirm the directory was created by running:
ls

You should see src, node_modules, package.json, and tsconfig.json in the output.

  • Create a temporary test file to verify tsx works by running:
vim src/test.ts
  • Enter insert mode by pressing i.
  • Type the following test code:
// Temporary test file to verify tsx works
const message: string = "tsx is working!";
console.log(message);
  • Save and exit Vim by pressing Esc, then typing :wq and pressing Enter.
  • Run the test file with tsx by running:
npx tsx src/test.ts

You should see tsx is working! printed in your terminal. This confirms TypeScript runs without a compile step.

Seeing an error instead of the message?

If you see a module or syntax error, check that your package.json has "type": "module" set. Run cat package.json to verify.

If the npx tsx command is not found, confirm the install succeeded by running ls node_modules/.bin/tsx. If it is missing, re-run npm install -D tsx.

Help me troubleshoot tsx

  • Clean up the test file since it is no longer needed by running:
rm src/test.ts

✔️ Awesome, I've got everything!

Great. Make sure you've saved both package.json and tsconfig.json.

ⓧ I'd like to double check the full code

Here is the complete package.json:

{
  "name": "ts-reviewer",
  "version": "1.0.0",
  "description": "TypeScript code review CLI for RLHF training",
  "type": "module",
  "scripts": {
    "start": "npx tsx src/index.ts",
    "review": "npx tsx src/index.ts",
    "review:easy": "npx tsx src/index.ts easy",
    "review:medium": "npx tsx src/index.ts medium",
    "review:hard": "npx tsx src/index.ts hard"
  },
  "devDependencies": {
    "typescript": "^6.0.3",
    "tsx": "^4.22.3"
  }
}

Here is the complete tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "outDir": "./dist",
    "rootDir": "./src"
  },
  "include": ["src/**/*"]
}

Your project foundation is ready. Next up, you'll build the bank of code snippet challenges that your review engine will present.

Build the Code Snippet Bank

Your CLI tool needs code to review. In a real RLHF workflow, you'd receive pairs of code snippets and decide which one is better.

In this step, you'll build a bank of challenge pairs that cover the exact issue categories G2i tests for: type safety problems, performance traps, security vulnerabilities, and logic errors. Each pair contains a weaker snippet and a stronger one.

In this step, get ready to:
  • Define TypeScript types for code challenges.
  • Write 4 challenge pairs covering common TypeScript pitfalls.
  • Export functions to retrieve challenges by difficulty.
Define TypeScript types for code challenges

Every challenge needs a consistent shape. Defining interfaces first gives you compile-time guarantees that each challenge object has all the required fields.

  • Create a new file src/snippets.ts inside your ts-reviewer project folder by running:
touch src/snippets.ts
  • Open src/snippets.ts in Vim by running:
vim src/snippets.ts
  • Add the following interface definitions at the top of the file:
export interface CodeSnippet {
  code: string;
  label: string;
}

export interface Challenge {
  id: string;
  title: string;
  description: string;
  difficulty: "easy" | "medium" | "hard";
  snippetA: CodeSnippet;
  snippetB: CodeSnippet;
  betterSnippet: "A" | "B";
  issues: string[];
}

What do these types do?

  • CodeSnippet holds a code string and a display label for each snippet in a pair.
  • Challenge defines the full shape of a review challenge: two snippets, which one is better, a difficulty rating, and a list of issues in the weaker snippet.
  • The difficulty field uses a union type so only "easy", "medium", or "hard" are valid values.

You won't see a change yet. The next chunk uses these types to define the actual challenge data.

Write 4 code challenge pairs

Each challenge presents two implementations of the same function. One has real-world issues that you'd flag in a code review. The four categories mirror what RLHF roles test: unsafe types, performance traps, security holes, and logic errors.

  • Below the interfaces in src/snippets.ts, add the challenges array:
const challenges: Challenge[] = [
  {
    id: "type-safety-01",
    title: "User Lookup Function",
    description:
      "Two implementations of a function that finds a user by ID from a list.",
    difficulty: "easy",
    snippetA: {
      code: `function findUser(users: any[], id: any): any {
  return users.find((u: any) => u.id === id);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface User {
  id: string;
  name: string;
  email: string;
}

function findUser(users: User[], id: string): User | undefined {
  return users.find((user) => user.id === id);
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "any type defeats TypeScript's purpose",
      "no return type safety",
      "caller has no IDE autocompletion",
      "runtime errors not caught at compile time",
    ],
  },
  {
    id: "performance-01",
    title: "Data Transformation Pipeline",
    description:
      "Two approaches to filtering and transforming a large array of transaction records.",
    difficulty: "medium",
    snippetA: {
      code: `interface Transaction {
  id: string;
  amount: number;
  status: "pending" | "completed" | "failed";
  timestamp: number;
}

function getCompletedTotals(transactions: Transaction[]): number[] {
  return transactions
    .filter((t) => t.status === "completed")
    .map((t) => t.amount)
    .filter((amount) => amount > 0)
    .sort((a, b) => b - a);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface Transaction {
  id: string;
  amount: number;
  status: "pending" | "completed" | "failed";
  timestamp: number;
}

function getCompletedTotals(transactions: Transaction[]): number[] {
  const results: number[] = [];

  for (const t of transactions) {
    if (t.status === "completed" && t.amount > 0) {
      results.push(t.amount);
    }
  }

  results.sort((a, b) => b - a);
  return results;
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "multiple array iterations instead of single pass",
      "intermediate array allocations on each chain step",
      "O(3n) filtering vs O(n) single loop",
      "readability is debatable but performance is not for large datasets",
    ],
  },
  {
    id: "security-01",
    title: "Query Parameter Handler",
    description:
      "Two implementations of a function that builds a database query from user input.",
    difficulty: "medium",
    snippetA: {
      code: `function buildQuery(userInput: string): string {
  return \`SELECT * FROM users WHERE name = '\${userInput}'\`;
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface QueryResult {
  text: string;
  params: string[];
}

function buildQuery(userInput: string): QueryResult {
  const sanitized = userInput.replace(/[^a-zA-Z0-9\\s\\-]/g, "");

  return {
    text: "SELECT * FROM users WHERE name = $1",
    params: [sanitized],
  };
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "SQL injection vulnerability via string interpolation",
      "no input validation or sanitization",
      "parameterized queries prevent injection attacks",
      "return type lacks structure for prepared statement use",
    ],
  },
  {
    id: "logic-01",
    title: "Pagination Calculator",
    description:
      "Two implementations of a function that calculates paginated slice boundaries.",
    difficulty: "hard",
    snippetA: {
      code: `function paginate<T>(items: T[], page: number, pageSize: number): T[] {
  const start = page * pageSize;
  const end = start + pageSize;
  return items.slice(start, end);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `function paginate<T>(items: T[], page: number, pageSize: number): T[] {
  if (page < 1 || pageSize < 1) {
    return [];
  }

  const start = (page - 1) * pageSize;
  const end = Math.min(start + pageSize, items.length);
  return items.slice(start, end);
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "zero-indexed page causes off-by-one for API consumers expecting page 1",
      "no validation for negative or zero page/pageSize",
      "no bounds checking could return empty arrays silently",
      "Math.min prevents reading past array bounds",
    ],
  },
];

What issues do these challenges cover?

  • type-safety-01 (easy): Using any everywhere vs proper interfaces. The any version compiles fine but gives zero help from the type system.
  • performance-01 (medium): Chaining .filter().map().filter().sort() creates multiple intermediate arrays vs a single-pass loop.
  • security-01 (medium): String interpolation into SQL vs parameterized queries with input sanitization.
  • logic-01 (hard): Zero-indexed pagination with no bounds checking vs one-indexed with validation.

You won't see a change yet. The export functions in the next section make these challenges accessible to the rest of the app.

Seeing a syntax error when you save?

The template literals inside the code properties use backticks (`). Make sure you haven't accidentally used regular quotes.

In the security challenge, the escaped backticks (\`) and escaped dollar signs (\${) are intentional. They prevent the template literal from being interpreted at the wrong level.

help me fix a syntax error in my snippets.ts challenges array

Export functions to retrieve challenges

The rest of the app needs a way to fetch challenges. One function returns all challenges (optionally filtered by difficulty), and another finds a specific challenge by ID.

  • At the bottom of src/snippets.ts, add these two export functions below the array:
export function getChallenges(difficulty?: Challenge["difficulty"]): Challenge[] {
  if (difficulty) {
    return challenges.filter((c) => c.difficulty === difficulty);
  }
  return challenges;
}

export function getChallengeById(id: string): Challenge | undefined {
  return challenges.find((c) => c.id === id);
}

What do these functions do?

  • getChallenges() returns all 4 challenges when called with no argument. Pass a difficulty like "medium" to filter down to matching challenges only.
  • getChallengeById() looks up a single challenge by its id field. It returns undefined if no match is found.
  • The Challenge["difficulty"] type annotation reuses the union type from the interface. This means you can only pass valid difficulty values.
  • Save src/snippets.ts in Vim by pressing Esc, then typing :wq and pressing Enter.
  • Verify the file compiles without errors by running:
npx tsx src/snippets.ts

You should see no output at all. No output means no TypeScript errors.

  • Confirm the exports work by running this quick check:
npx tsx -e "import { getChallenges } from './src/snippets.js'; console.log(getChallenges().length + ' challenges loaded');"

You should see 4 challenges loaded printed in the terminal.

Seeing a type error or module not found?

If you see a module resolution error, make sure your package.json has "type": "module" set. The .js extension in the import path is required by NodeNext module resolution. tsx resolves it to the .ts file automatically.

If you see a type error, check that your interfaces match the challenge objects exactly. The difficulty field must be one of the three allowed string values, not a generic string.

help me fix a module resolution or type error in my snippets.ts file

✔️ Awesome, I've got everything!

Great. Make sure you've saved src/snippets.ts and confirmed the 4 challenges loaded output before moving on.

ⓧ I'd like to double check the full code

Here is the complete src/snippets.ts file. Compare your version against this:

export interface CodeSnippet {
  code: string;
  label: string;
}

export interface Challenge {
  id: string;
  title: string;
  description: string;
  difficulty: "easy" | "medium" | "hard";
  snippetA: CodeSnippet;
  snippetB: CodeSnippet;
  betterSnippet: "A" | "B";
  issues: string[];
}

const challenges: Challenge[] = [
  {
    id: "type-safety-01",
    title: "User Lookup Function",
    description:
      "Two implementations of a function that finds a user by ID from a list.",
    difficulty: "easy",
    snippetA: {
      code: `function findUser(users: any[], id: any): any {
  return users.find((u: any) => u.id === id);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface User {
  id: string;
  name: string;
  email: string;
}

function findUser(users: User[], id: string): User | undefined {
  return users.find((user) => user.id === id);
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "any type defeats TypeScript's purpose",
      "no return type safety",
      "caller has no IDE autocompletion",
      "runtime errors not caught at compile time",
    ],
  },
  {
    id: "performance-01",
    title: "Data Transformation Pipeline",
    description:
      "Two approaches to filtering and transforming a large array of transaction records.",
    difficulty: "medium",
    snippetA: {
      code: `interface Transaction {
  id: string;
  amount: number;
  status: "pending" | "completed" | "failed";
  timestamp: number;
}

function getCompletedTotals(transactions: Transaction[]): number[] {
  return transactions
    .filter((t) => t.status === "completed")
    .map((t) => t.amount)
    .filter((amount) => amount > 0)
    .sort((a, b) => b - a);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface Transaction {
  id: string;
  amount: number;
  status: "pending" | "completed" | "failed";
  timestamp: number;
}

function getCompletedTotals(transactions: Transaction[]): number[] {
  const results: number[] = [];

  for (const t of transactions) {
    if (t.status === "completed" && t.amount > 0) {
      results.push(t.amount);
    }
  }

  results.sort((a, b) => b - a);
  return results;
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "multiple array iterations instead of single pass",
      "intermediate array allocations on each chain step",
      "O(3n) filtering vs O(n) single loop",
      "readability is debatable but performance is not for large datasets",
    ],
  },
  {
    id: "security-01",
    title: "Query Parameter Handler",
    description:
      "Two implementations of a function that builds a database query from user input.",
    difficulty: "medium",
    snippetA: {
      code: `function buildQuery(userInput: string): string {
  return \`SELECT * FROM users WHERE name = '\${userInput}'\`;
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `interface QueryResult {
  text: string;
  params: string[];
}

function buildQuery(userInput: string): QueryResult {
  const sanitized = userInput.replace(/[^a-zA-Z0-9\\s\\-]/g, "");

  return {
    text: "SELECT * FROM users WHERE name = $1",
    params: [sanitized],
  };
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "SQL injection vulnerability via string interpolation",
      "no input validation or sanitization",
      "parameterized queries prevent injection attacks",
      "return type lacks structure for prepared statement use",
    ],
  },
  {
    id: "logic-01",
    title: "Pagination Calculator",
    description:
      "Two implementations of a function that calculates paginated slice boundaries.",
    difficulty: "hard",
    snippetA: {
      code: `function paginate<T>(items: T[], page: number, pageSize: number): T[] {
  const start = page * pageSize;
  const end = start + pageSize;
  return items.slice(start, end);
}`,
      label: "Snippet A",
    },
    snippetB: {
      code: `function paginate<T>(items: T[], page: number, pageSize: number): T[] {
  if (page < 1 || pageSize < 1) {
    return [];
  }

  const start = (page - 1) * pageSize;
  const end = Math.min(start + pageSize, items.length);
  return items.slice(start, end);
}`,
      label: "Snippet B",
    },
    betterSnippet: "B",
    issues: [
      "zero-indexed page causes off-by-one for API consumers expecting page 1",
      "no validation for negative or zero page/pageSize",
      "no bounds checking could return empty arrays silently",
      "Math.min prevents reading past array bounds",
    ],
  },
];

export function getChallenges(difficulty?: Challenge["difficulty"]): Challenge[] {
  if (difficulty) {
    return challenges.filter((c) => c.difficulty === difficulty);
  }
  return challenges;
}

export function getChallengeById(id: string): Challenge | undefined {
  return challenges.find((c) => c.id === id);
}

Your snippet bank is ready. Next up, you'll build the review engine that presents these challenges and collects your rankings and written explanations.

Build the Review Engine

You have a bank of code challenges ready to go. Now you need the engine that actually presents those challenges to a reviewer and collects structured feedback.

The review engine is the heart of the RLHF workflow. It guides you through each step of the process: read two snippets, rank them, explain your reasoning, then produce a refactored version. Each piece of feedback gets captured as structured data.

In this step, get ready to:
  • Create the core review loop that presents challenges and collects rankings.
  • Build input collection functions using Node.js readline promises.
  • Add a refactoring prompt that captures improved code from the reviewer.
Create the review loop

The review loop orchestrates the entire interaction for a single challenge. It displays both snippets side by side, then walks the reviewer through ranking, justification, and refactoring.

  • Create a new file src/reviewer.ts in Vim by running:
vim src/reviewer.ts
  • Add the imports, the ReviewResult interface, and the main reviewChallenge function by pasting this code:
import * as readline from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";
import { Challenge } from "./snippets.js";

export interface ReviewResult {
  challengeId: string;
  chosenBetter: "A" | "B";
  justification: string;
  refactoredCode: string;
  timestamp: string;
}

export async function reviewChallenge(
  challenge: Challenge
): Promise<ReviewResult> {
  const rl = readline.createInterface({ input, output });

  console.log("\n" + "=".repeat(60));
  console.log(`Challenge: ${challenge.title}`);
  console.log(`Difficulty: ${challenge.difficulty}`);
  console.log(`Description: ${challenge.description}`);
  console.log("=".repeat(60));

  console.log("\n--- Snippet A ---");
  console.log(challenge.snippetA.code);

  console.log("\n--- Snippet B ---");
  console.log(challenge.snippetB.code);

  console.log("\n" + "-".repeat(60));

  const ranking = await askRanking(rl);
  const justification = await askJustification(rl);
  const refactoredCode = await askRefactoring(rl);

  rl.close();

  return {
    challengeId: challenge.id,
    chosenBetter: ranking,
    justification,
    refactoredCode,
    timestamp: new Date().toISOString(),
  };
}

What does this code do?

This is the orchestrator for a single review challenge. It handles three responsibilities:

  • ReviewResult defines the exact shape of structured feedback. Every review produces a ranking, justification, refactored code, and timestamp.
  • reviewChallenge creates a readline interface, prints both snippets with clear visual separators, then calls three helper functions in sequence.
  • The import path ./snippets.js uses the .js extension because NodeNext module resolution requires it. tsx handles the mapping to the actual .ts file automatically.

Why node:readline/promises?

The node:readline/promises API provides an async interface for terminal input. Instead of callbacks, you can await each question. This keeps the review flow linear and readable.

Implement ranking and justification input

The first two helper functions handle the core RLHF inputs: which snippet is better, and why. The ranking uses simple validation. The justification collects multi-line text.

  • Add the askRanking function below the reviewChallenge function:
async function askRanking(
  rl: readline.Interface
): Promise<"A" | "B"> {
  while (true) {
    const answer = await rl.question(
      "\nWhich snippet is better? (A or B): "
    );
    const normalized = answer.trim().toUpperCase();
    if (normalized === "A" || normalized === "B") {
      return normalized;
    }
    console.log("Please enter A or B.");
  }
}

What does this code do?

  • The while (true) loop keeps asking until the user enters a valid response. This is input validation in action.
  • The return type Promise<"A" | "B"> is a union literal type. TypeScript guarantees this function can only ever return one of those two exact strings.
  • Add the askJustification function below askRanking:
async function askJustification(
  rl: readline.Interface
): Promise<string> {
  console.log(
    "\nExplain WHY your choice is better. What issues does the other have?"
  );
  console.log("(Write your explanation, then press Enter twice to finish)");

  const lines: string[] = [];
  let emptyLineCount = 0;

  while (true) {
    const line = await rl.question("");
    if (line === "") {
      emptyLineCount++;
      if (emptyLineCount >= 1 && lines.length > 0) {
        break;
      }
    } else {
      emptyLineCount = 0;
      lines.push(line);
    }
  }

  return lines.join("\n");
}

What does this code do?

  • This collects multi-line text input from the reviewer. Each line gets pushed into an array until the user presses Enter on an empty line.
  • The lines.length > 0 check prevents an accidental empty Enter at the start from ending input prematurely.
  • Finally, lines.join("\n") reassembles the array into a single string, preserving the paragraph structure of the reviewer's explanation.

Why does this matter for RLHF?

The justification prompt mirrors exactly what G2i tests for: explaining WHY one approach is better than another. The prompt explicitly asks about issues in the weaker snippet because that's the skill RLHF roles demand. You're not just saying "B is better". You're articulating the technical reasoning.

Add the refactoring prompt

The final piece of the RLHF loop is refactoring. After ranking and explaining, the reviewer writes an improved version of the weaker snippet. This exercises the "repair and refactor" skill.

  • Add the askRefactoring function below askJustification:
async function askRefactoring(
  rl: readline.Interface
): Promise<string> {
  console.log(
    "\nWrite a refactored version of the WEAKER snippet."
  );
  console.log("(Paste your code, then press Enter twice to finish)");

  const lines: string[] = [];
  let emptyLineCount = 0;

  while (true) {
    const line = await rl.question("");
    if (line === "") {
      emptyLineCount++;
      if (emptyLineCount >= 1 && lines.length > 0) {
        break;
      }
    } else {
      emptyLineCount = 0;
      lines.push(line);
    }
  }

  return lines.join("\n");
}

What does this code do?

This function uses the same multi-line collection pattern as askJustification, but the prompt asks for code instead of prose. The reviewer pastes their refactored version of the weaker snippet, and it gets stored as a string in the review record.

  • Save src/reviewer.ts in Vim by pressing Esc, then typing :wq and pressing Enter.
  • Verify TypeScript is happy with your code by running:
npx tsc --noEmit

If there are no errors, the command completes silently with no output. That silence means your types are correct.

Seeing type errors?

  • Check that the import path says ./snippets.js (with the .js extension), not ./snippets.ts. NodeNext resolution requires the .js extension in imports.
  • Make sure the Challenge import matches the exported interface name in src/snippets.ts exactly.
  • Confirm that node:readline/promises uses the node: prefix. Without it, TypeScript may not resolve the module correctly.

Help me debug TypeScript errors in my reviewer.ts file

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

import * as readline from "node:readline/promises";
import { stdin as input, stdout as output } from "node:process";
import { Challenge } from "./snippets.js";

export interface ReviewResult {
  challengeId: string;
  chosenBetter: "A" | "B";
  justification: string;
  refactoredCode: string;
  timestamp: string;
}

export async function reviewChallenge(
  challenge: Challenge
): Promise<ReviewResult> {
  const rl = readline.createInterface({ input, output });

  console.log("\n" + "=".repeat(60));
  console.log(`Challenge: ${challenge.title}`);
  console.log(`Difficulty: ${challenge.difficulty}`);
  console.log(`Description: ${challenge.description}`);
  console.log("=".repeat(60));

  console.log("\n--- Snippet A ---");
  console.log(challenge.snippetA.code);

  console.log("\n--- Snippet B ---");
  console.log(challenge.snippetB.code);

  console.log("\n" + "-".repeat(60));

  const ranking = await askRanking(rl);
  const justification = await askJustification(rl);
  const refactoredCode = await askRefactoring(rl);

  rl.close();

  return {
    challengeId: challenge.id,
    chosenBetter: ranking,
    justification,
    refactoredCode,
    timestamp: new Date().toISOString(),
  };
}

async function askRanking(
  rl: readline.Interface
): Promise<"A" | "B"> {
  while (true) {
    const answer = await rl.question(
      "\nWhich snippet is better? (A or B): "
    );
    const normalized = answer.trim().toUpperCase();
    if (normalized === "A" || normalized === "B") {
      return normalized;
    }
    console.log("Please enter A or B.");
  }
}

async function askJustification(
  rl: readline.Interface
): Promise<string> {
  console.log(
    "\nExplain WHY your choice is better. What issues does the other have?"
  );
  console.log("(Write your explanation, then press Enter twice to finish)");

  const lines: string[] = [];
  let emptyLineCount = 0;

  while (true) {
    const line = await rl.question("");
    if (line === "") {
      emptyLineCount++;
      if (emptyLineCount >= 1 && lines.length > 0) {
        break;
      }
    } else {
      emptyLineCount = 0;
      lines.push(line);
    }
  }

  return lines.join("\n");
}

async function askRefactoring(
  rl: readline.Interface
): Promise<string> {
  console.log(
    "\nWrite a refactored version of the WEAKER snippet."
  );
  console.log("(Paste your code, then press Enter twice to finish)");

  const lines: string[] = [];
  let emptyLineCount = 0;

  while (true) {
    const line = await rl.question("");
    if (line === "") {
      emptyLineCount++;
      if (emptyLineCount >= 1 && lines.length > 0) {
        break;
      }
    } else {
      emptyLineCount = 0;
      lines.push(line);
    }
  }

  return lines.join("\n");
}

Your review engine is ready. It can present challenges, validate input, and collect structured feedback in the exact format RLHF pipelines expect. Next up, you'll build the recorder that persists this feedback to a JSON file.

Build the Feedback Recorder

Your review engine can now collect rankings, justifications, and refactored code from the user. But without persistence, all that work disappears the moment the program exits.

In this step, you'll build a recorder module that saves each review to a JSON file on disk. This creates the structured output that proves your code review skills to potential employers.

In this step, get ready to:
  • Define the output schema for review sessions.
  • Implement file I/O to persist and retrieve review data.
Define the output schema

Before writing any logic, you need a clear data contract. The recorder needs to know what shape a review session takes so it can read and write JSON reliably.

  • Create a new file called src/recorder.ts in Vim by running:
vim src/recorder.ts
  • Add the imports, interface, and output file constant by pasting this code:
import { readFile, writeFile } from "node:fs/promises";
import { existsSync } from "node:fs";
import { ReviewResult } from "./reviewer.js";

export interface ReviewSession {
  sessionId: string;
  startedAt: string;
  reviews: ReviewResult[];
}

const OUTPUT_FILE = "reviews.json";

What does this code do?

The imports pull in Node.js file system utilities. readFile and writeFile are async (promise-based), while existsSync is synchronous for a quick file-existence check.

The ReviewSession interface wraps multiple reviews into a single session with a unique ID and timestamp. The OUTPUT_FILE constant centralizes the filename so every function references the same path.

  • Save the file in Vim by pressing Esc, then typing :w and pressing Enter.
  • Verify the file compiles by running:
npx tsx src/recorder.ts

The command should exit cleanly with no output. That confirms TypeScript is happy with your imports and types.

Seeing a module not found error?

Check that src/reviewer.ts exists and exports the ReviewResult interface. The import path uses ./reviewer.js (with the .js extension) because NodeNext module resolution requires it.

help me debug a module not found error in my recorder.ts file

Implement the session loader

The recorder needs to handle two scenarios: starting a fresh session when no file exists, and resuming an existing session when one does. This prevents data loss between runs.

  • In src/recorder.ts, add the loadSession function below the OUTPUT_FILE constant:
export async function loadSession(): Promise<ReviewSession> {
  if (existsSync(OUTPUT_FILE)) {
    const data = await readFile(OUTPUT_FILE, "utf-8");
    return JSON.parse(data) as ReviewSession;
  }

  return {
    sessionId: crypto.randomUUID(),
    startedAt: new Date().toISOString(),
    reviews: [],
  };
}

What does this code do?

  • existsSync(OUTPUT_FILE) checks if reviews.json already exists on disk.
  • If the file exists, it reads and parses it back into a ReviewSession object.
  • If no file exists, it creates a brand new session with a unique ID generated by crypto.randomUUID().
  • Save the file.

You won't see a change yet. The next functions will use loadSession to read and write review data.

  • Add the saveReview function below loadSession:
export async function saveReview(review: ReviewResult): Promise<void> {
  const session = await loadSession();
  session.reviews.push(review);
  await writeFile(OUTPUT_FILE, JSON.stringify(session, null, 2), "utf-8");
}

What does this code do?

Each call to saveReview loads the current session, appends the new review, and writes the entire session back to disk. The JSON.stringify(session, null, 2) call pretty-prints the output with 2-space indentation so you can read the file easily.

  • Save the file.

You won't see a change yet. The next function completes the recorder's public API.

  • Add the printSummary function below saveReview:
export async function printSummary(): Promise<void> {
  const session = await loadSession();
  console.log("\n" + "=".repeat(60));
  console.log("SESSION SUMMARY");
  console.log("=".repeat(60));
  console.log(`Reviews completed: ${session.reviews.length}`);
  console.log(`Session ID: ${session.sessionId}`);
  console.log(`Output saved to: ${OUTPUT_FILE}`);
  console.log("=".repeat(60));
}

What does this code do?

printSummary gives the user a quick recap after completing their reviews. It shows how many reviews were recorded, the session's unique ID, and where the output file lives.

  • Save src/recorder.ts and exit Vim by pressing Esc, then typing :wq and pressing Enter.
  • Confirm the complete file compiles by running:
npx tsx src/recorder.ts

The command should exit cleanly with no output and no errors. This confirms all your types, imports, and function signatures are correct.

Seeing a type error?

Make sure every function has the correct return type annotation. loadSession returns Promise<ReviewSession>, while saveReview and printSummary return Promise<void>.

Also check that the ReviewResult import path ends in .js (not .ts). NodeNext resolution requires the compiled extension in import paths.

help me fix a type error in my recorder.ts file

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete src/recorder.ts file:

import { readFile, writeFile } from "node:fs/promises";
import { existsSync } from "node:fs";
import { ReviewResult } from "./reviewer.js";

export interface ReviewSession {
  sessionId: string;
  startedAt: string;
  reviews: ReviewResult[];
}

const OUTPUT_FILE = "reviews.json";

export async function loadSession(): Promise<ReviewSession> {
  if (existsSync(OUTPUT_FILE)) {
    const data = await readFile(OUTPUT_FILE, "utf-8");
    return JSON.parse(data) as ReviewSession;
  }

  return {
    sessionId: crypto.randomUUID(),
    startedAt: new Date().toISOString(),
    reviews: [],
  };
}

export async function saveReview(review: ReviewResult): Promise<void> {
  const session = await loadSession();
  session.reviews.push(review);
  await writeFile(OUTPUT_FILE, JSON.stringify(session, null, 2), "utf-8");
}

export async function printSummary(): Promise<void> {
  const session = await loadSession();
  console.log("\n" + "=".repeat(60));
  console.log("SESSION SUMMARY");
  console.log("=".repeat(60));
  console.log(`Reviews completed: ${session.reviews.length}`);
  console.log(`Session ID: ${session.sessionId}`);
  console.log(`Output saved to: ${OUTPUT_FILE}`);
  console.log("=".repeat(60));
}

Your recorder is ready to persist every review to disk. Next up, you'll wire the snippet bank, review engine, and recorder together into a single CLI that runs the full RLHF workflow end to end.

Wire It All Together as a CLI

You've built all the individual pieces of the review tool. The snippet bank holds your challenges, the reviewer collects rankings and justifications, and the recorder saves everything to disk.

Now you need an entry point that orchestrates these modules into a single CLI flow. In this step, you'll create index.ts to wire everything together, add difficulty filtering via command-line arguments, and run through all four review challenges.

In this step, get ready to:
  • Create the main entry point that orchestrates the full review flow.
  • Add a command-line argument parser for difficulty filtering.
  • Run the complete tool and practice your written code review explanations.
Create the main entry point

The entry point file imports all your modules and calls them in sequence. It also handles argument parsing so you can filter challenges by difficulty directly from the terminal.

  • Create a new file src/index.ts and add the following code:
import { getChallenges, Challenge } from "./snippets.js";
import { reviewChallenge } from "./reviewer.js";
import { saveReview, printSummary } from "./recorder.js";

async function main(): Promise<void> {
  const args = process.argv.slice(2);
  const difficultyArg = args.find((a) =>
    ["easy", "medium", "hard"].includes(a)
  ) as Challenge["difficulty"] | undefined;

  console.log("╔══════════════════════════════════════════╗");
  console.log("║       ts-reviewer: RLHF Training        ║");
  console.log("╠══════════════════════════════════════════╣");
  console.log("║  Compare, Rank, Explain, Refactor       ║");
  console.log("╚══════════════════════════════════════════╝");

  if (difficultyArg) {
    console.log(`\nFilter: ${difficultyArg} challenges only`);
  }

  const challenges = getChallenges(difficultyArg);

  if (challenges.length === 0) {
    console.log("No challenges found for the selected difficulty.");
    process.exit(1);
  }

  console.log(`\nLoaded ${challenges.length} challenge(s). Let's begin.\n`);

  for (const challenge of challenges) {
    const result = await reviewChallenge(challenge);
    await saveReview(result);
    console.log("\n✓ Review saved.");
  }

  await printSummary();
}

main().catch((error: unknown) => {
  console.error("Error:", error);
  process.exit(1);
});

What does this code do?

This file is the orchestrator for your entire CLI tool. Here's how each section works:

  • process.argv.slice(2) grabs any arguments the user passes after the script name. If someone runs npx tsx src/index.ts easy, the args array would contain ["easy"].
  • difficultyArg searches for a valid difficulty level in the arguments. If none is found, it stays undefined and all challenges load.
  • The for...of loop runs each challenge sequentially, awaiting the user's input before moving to the next one.
  • main().catch() wraps the entire flow in error handling so any unexpected crash prints a useful message instead of a raw stack trace.
  • Save src/index.ts.

Seeing import errors?

Make sure your import paths end with .js (not .ts). This is required by NodeNext module resolution. tsx handles resolving them to the actual .ts files at runtime.

Why won't my imports resolve correctly?

Use the CLI argument parser

The argument parser you just wrote lets you filter challenges by difficulty directly from the command line. This means you can practice one category at a time or run everything at once.

How does the difficulty filter work?

When you pass a difficulty level as a CLI argument, getChallenges(difficultyArg) filters the snippet bank to only return matching challenges. Without an argument, it returns all four.

Your package.json already has convenience scripts set up for this. You can run npx tsx src/index.ts easy to tackle just the type safety challenge, or npx tsx src/index.ts medium for the performance and security challenges.

Run the complete tool

Now it's time to use your tool for real. You'll run all four challenges and practice the exact RLHF workflow: compare, rank, explain, and refactor.

  • Launch the review tool by running:
npx tsx src/index.ts

You should see the ASCII banner followed by the first challenge (User Lookup Function).

What should I see?

The tool prints two code snippets side by side and prompts you with three questions in sequence:

  • Which snippet is better? (A or B) Pick the one with fewer issues.
  • Explain WHY your choice is better. Write a clear, specific justification. Press Enter twice when done.
  • Write a refactored version. Paste or type an improved version of the weaker snippet. Press Enter twice when done.

Write like you're training an AI

In your justifications, be specific about the problem. Don't just say "Snippet B is better." Name the exact issue: "Snippet A uses any types which bypass compile-time checks and remove IDE autocompletion."

This is the exact style of writing that RLHF roles demand. The clearer your explanation, the better an AI can learn from it.

  • Complete all 4 review challenges, writing detailed justifications for each one.
  • After the final challenge, the tool prints a session summary showing how many reviews you completed.
  • Confirm that a reviews.json file was created in your project folder by running:
ls

You should see reviews.json in the output alongside your src folder, package.json, and tsconfig.json.

Tool exits immediately or shows an error?

If you see a "Cannot find module" error, check that all four files exist in your src/ folder: index.ts, snippets.ts, reviewer.ts, and recorder.ts. Also verify your package.json has "type": "module" set.

Help me debug why my ts-reviewer CLI won't start.

✔️ Awesome, I've got everything!

Great. Double check you've saved src/index.ts and confirmed that reviews.json was generated.

ⓧ I'd like to double check the full code

Here's the complete src/index.ts file:

import { getChallenges, Challenge } from "./snippets.js";
import { reviewChallenge } from "./reviewer.js";
import { saveReview, printSummary } from "./recorder.js";

async function main(): Promise<void> {
  const args = process.argv.slice(2);
  const difficultyArg = args.find((a) =>
    ["easy", "medium", "hard"].includes(a)
  ) as Challenge["difficulty"] | undefined;

  console.log("╔══════════════════════════════════════════╗");
  console.log("║       ts-reviewer: RLHF Training        ║");
  console.log("╠══════════════════════════════════════════╣");
  console.log("║  Compare, Rank, Explain, Refactor       ║");
  console.log("╚══════════════════════════════════════════╝");

  if (difficultyArg) {
    console.log(`\nFilter: ${difficultyArg} challenges only`);
  }

  const challenges = getChallenges(difficultyArg);

  if (challenges.length === 0) {
    console.log("No challenges found for the selected difficulty.");
    process.exit(1);
  }

  console.log(`\nLoaded ${challenges.length} challenge(s). Let's begin.\n`);

  for (const challenge of challenges) {
    const result = await reviewChallenge(challenge);
    await saveReview(result);
    console.log("\n✓ Review saved.");
  }

  await printSummary();
}

main().catch((error: unknown) => {
  console.error("Error:", error);
  process.exit(1);
});

Your CLI tool is complete. You now have a working TypeScript application that walks you through the full RLHF code review workflow, and a reviews.json file full of your written justifications. Next up, you'll add a scoring rubric that evaluates how thorough your reviews are.

Secret mission

Add a Scoring Rubric

Your tool collects reviews, but how do you know if you're actually catching the right issues? Add a self-scoring rubric that evaluates your justifications against expected keywords and tells you what you missed.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • The ts-reviewer project folder on your Desktop (includes all source files, node_modules, and tsconfig.json).
  • The reviews.json output file generated when you ran the tool (inside the project folder).

Keep everything running

No action needed. Choose this if you plan to keep practicing your code review skills or want to add more challenges to the snippet bank.

  • Your ts-reviewer folder stays on your Desktop with all source files intact.
  • Your reviews.json file keeps your review history for future --score evaluations.
  • Run npx tsx src/index.ts any time to practice more reviews.

Pause - I'll come back to this later

Since this project is entirely local and has no running processes, there is nothing to pause. The files sit dormant on your Desktop until you run the tool again.

  • Your project folder takes up minimal disk space (the node_modules directory is the largest part).
  • If you want to free up space without losing your code, delete node_modules and reinstall later with npm install.

Delete - I don't want to use this again

Remove all project files from your machine. This deletes your source code, dependencies, and review history permanently.

  • Delete the entire project folder by running this command (macOS/Linux):
rm -rf ~/Desktop/ts-reviewer
  • On Windows, run this instead:
rmdir /s /q %USERPROFILE%\Desktop\ts-reviewer
  • This removes all source files, node_modules, reviews.json, and configuration in one step.

Nice Work!

Nice Work!

Nice work. You just built a TypeScript CLI tool that directly trains the skills tested in RLHF code review roles.

You've learned how to:

  • Build a complete TypeScript CLI application using modern tooling (tsx, strict TypeScript) that presents code challenges and collects structured feedback.
  • Create a bank of realistic code review challenges covering the exact issue categories RLHF roles test for: logic errors, performance traps, and security vulnerabilities.
  • Practice writing structured technical justifications through the full RLHF feedback loop: rank, explain, and refactor.
  • Secret Mission: Add a self-scoring rubric system that evaluates your written justifications against expected keywords and highlights blind spots in your code review instincts.

Ready to quiz yourself?