Build a Security Review Skill for Claude Code
Build a custom /security-review skill that scans code changes for vulnerabilities.
Introduction
30 Second Summary
Security vulnerabilities slip into code every day. A missed hardcoded key, an unsanitized input, a plaintext password stored in the clear. By the time a human reviewer catches them, the damage is already in your commit history.
In this project, you will build a custom Claude Code skill that automatically reviews your code changes for security vulnerabilities before you commit them.
What You'll Build
Type /security-review in your terminal and watch Claude scan your uncommitted changes, flagging hardcoded secrets, injection risks, and weak authentication patterns in a structured, prioritized report you can act on immediately.
By the end of this project, you'll have:
- A working /security-review skill that inspects your Git diff for five vulnerability categories and returns prioritized findings with concrete fix suggestions.
- A CLAUDE.md project memory file that teaches Claude your security standards, so every review is consistent and thorough.
- An automated test script that plants known vulnerabilities and verifies the skill catches all five categories, proving it works reliably.
- Secret Mission: Wire the skill to review only staged changes or branch-specific diffs for pre-commit and PR review workflows.
Are there any prerequisites?
You need Claude Code installed with an active subscription, Node.js (v18+), and Git on your machine. Familiarity with TypeScript basics is helpful but not required.
Before We Start
Before we dive in, let's lock in what you're about to build and why it matters for your day-to-day development workflow.
Set Up the Vulnerable Sample Project
Your security review skill needs something to review. Without real code containing real vulnerabilities, you have no way to verify the skill actually catches anything.
In this step, you will create a small TypeScript project with deliberately insecure code planted across four files. Each file targets a different vulnerability category: hardcoded secrets, SQL injection, weak authentication, and missing input validation.
In this step, get ready to:
- Scaffold a TypeScript project with Git version control.
- Create four source files containing deliberate security vulnerabilities.
- Verify the code compiles cleanly despite being insecure.
Create and initialize the project
- Move to your Desktop by running this command:
cd ~/Desktop- Create your project folder, move into it, and initialize a Git repository by running these commands:
mkdir secure-review-lab
cd secure-review-lab
git init- Create a package.json file in the secure-review-lab folder with the following content:
{
"name": "secure-review-lab",
"version": "1.0.0",
"description": "Intentionally vulnerable TypeScript app for testing Claude Code security review skill",
"scripts": {
"typecheck": "tsc --noEmit"
},
"devDependencies": {
"typescript": "5.5.4"
}
}What does this file do?
The package.json declares your project and its single dependency: TypeScript at version 5.5.4. The typecheck script gives you a quick way to verify the code compiles without producing output files.
- Create a tsconfig.json file in the same folder with the following content:
{
"compilerOptions": {
"target": "ES2022",
"module": "Node16",
"moduleResolution": "Node16",
"strict": true,
"esModuleInterop": true,
"outDir": "./dist",
"rootDir": "./src",
"skipLibCheck": true
},
"include": ["src/**/*"]
}What does this configure?
This tells TypeScript to use strict mode (which catches more type errors), target modern JavaScript, and look for source files in the src/ directory. Strict mode is important because it proves the vulnerable code is valid TypeScript that just happens to be insecure.
- Install TypeScript by running:
npm install- You should see output showing typescript being added to your node_modules folder.
- Make your first commit by running:
git add .
git commit -m "initial project setup"Seeing a git identity error?
If Git asks you to set your name and email before committing, run these two commands (replacing the placeholders with your info), then retry the commit:
git config user.email "you@example.com"
git config user.name "Your Name"help me fix my git commit error
Build the vulnerable source files
Now you will create four files that each demonstrate a different class of security vulnerability. These files are deliberately insecure so your skill has real issues to find.
- Create the src directory by running:
mkdir src- Create src/config.ts with the following content:
// DELIBERATELY VULNERABLE - for security review testing
export const config = {
database: {
host: "db.production.internal",
port: 5432,
username: "admin",
password: "SuperSecret123!",
name: "app_production",
},
api: {
stripeKey: "sk_live_4eC39HqLyjWDarjtT1zdp7dc",
sendgridKey: "SG.abc123def456ghi789jkl012mno345",
jwtSecret: "my-super-secret-jwt-key-do-not-share",
},
aws: {
accessKeyId: "AKIAIOSFODNN7EXAMPLE",
secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
region: "us-east-1",
},
};What vulnerabilities does this contain?
Every credential in this file is hardcoded directly in the source code: database passwords, API keys for Stripe and SendGrid, a JWT signing secret, and AWS access keys. In a real project, these would be environment variables or fetched from a secrets manager.
- Create src/database.ts and paste the following setup code at the top:
// DELIBERATELY VULNERABLE - for security review testing
import { config } from "./config.js";
interface QueryResult {
rows: Record<string, unknown>[];
}
function executeQuery(sql: string): QueryResult {
console.log(`Executing: ${sql}`);
return { rows: [] };
}- Below that code in the same file, add the query functions:
export function getUserByEmail(email: string): QueryResult {
const query = `SELECT * FROM users WHERE email = '${email}'`;
return executeQuery(query);
}
export function searchProducts(name: string, category: string): QueryResult {
const query = `SELECT * FROM products WHERE name LIKE '%${name}%' AND category = '${category}'`;
return executeQuery(query);
}
export function deleteUser(userId: string): QueryResult {
const query = `DELETE FROM users WHERE id = ${userId}`;
return executeQuery(query);
}
export function updateUserRole(userId: string, role: string): QueryResult {
const query = `UPDATE users SET role = '${role}' WHERE id = ${userId}`;
return executeQuery(query);
}What vulnerabilities does this contain?
Every query function builds SQL by directly interpolating user input into the string. An attacker could pass ' OR '1'='1 as an email to dump the entire users table, or inject a DROP TABLE into the delete function.
- Save src/database.ts and confirm your files exist by running:
ls src/- You should see config.ts and database.ts listed.
- Create src/auth.ts and paste the following interfaces and registration function:
// DELIBERATELY VULNERABLE - for security review testing
interface User {
id: string;
email: string;
password: string;
role: string;
}
interface Session {
token: string;
userId: string;
createdAt: number;
}
const users: User[] = [];
const sessions: Session[] = [];
export function registerUser(email: string, password: string): User {
const user: User = {
id: String(users.length + 1),
email,
password,
role: "user",
};
users.push(user);
return user;
}- Below that, add the session and validation functions in the same file:
export function loginUser(email: string, password: string): Session | null {
const user = users.find((u) => u.email === email && u.password === password);
if (!user) return null;
const session: Session = {
token: `session_${user.id}_${Date.now()}`,
userId: user.id,
createdAt: Date.now(),
};
sessions.push(session);
return session;
}
export function validateSession(token: string): User | null {
const session = sessions.find((s) => s.token === token);
if (!session) return null;
return users.find((u) => u.id === session.userId) || null;
}
export function isAdmin(user: User): boolean {
return user.role === "admin";
}What vulnerabilities does this contain?
The registerUser function stores the password in plaintext with no hashing. Login compares plaintext passwords directly.
Session tokens are predictable (session_1_1691234567890 pattern) and never expire. An attacker who guesses the format can forge valid session tokens.
- Create src/server.ts and paste the imports and body parser:
// DELIBERATELY VULNERABLE - for security review testing
import * as http from "node:http";
import { getUserByEmail, searchProducts } from "./database.js";
import { registerUser, loginUser, validateSession } from "./auth.js";
function parseBody(req: http.IncomingMessage): Promise<string> {
return new Promise((resolve) => {
let body = "";
req.on("data", (chunk) => (body += chunk));
req.on("end", () => resolve(body));
});
}- Below that, add the POST route handlers:
const server = http.createServer(async (req, res) => {
const url = new URL(req.url || "/", `http://${req.headers.host}`);
if (req.method === "POST") {
const body = await parseBody(req);
const data = JSON.parse(body);
if (url.pathname === "/register") {
const user = registerUser(data.email, data.password);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify(user));
return;
}
if (url.pathname === "/login") {
const session = loginUser(data.email, data.password);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify(session));
return;
}
}- Finally, add the GET route handlers and server listener at the bottom of the same file:
if (req.method === "GET") {
if (url.pathname === "/user") {
const email = url.searchParams.get("email") || "";
const result = getUserByEmail(email);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify(result));
return;
}
if (url.pathname === "/search") {
const name = url.searchParams.get("q") || "";
const category = url.searchParams.get("cat") || "";
const result = searchProducts(name, category);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify(result));
return;
}
}
res.writeHead(404);
res.end("Not found");
});
server.listen(3000, () => {
console.log("Server running on http://localhost:3000");
});What vulnerabilities does this contain?
The server has no input validation anywhere. There is no content-type check on incoming requests, no size limit on request bodies, no try/catch around JSON.parse, no email format validation on registration, and no password strength requirements.
Query parameters from GET requests pass directly to the SQL injection-vulnerable database functions with no sanitization.
- Save src/server.ts and confirm all four files exist by running:
ls src/- You should see auth.ts, config.ts, database.ts, and server.ts listed.
- Commit all source files by running:
git add .
git commit -m "add vulnerable source files"✔️ Awesome, I've got everything!
Great. Double check you have saved all four files in src/ and committed them.
ⓧ I'd like to double check the full code
Your project should have these files. Scroll up and confirm each section is present in order:
- package.json with the typescript devDependency and typecheck script.
- tsconfig.json with strict mode, Node16 module resolution, and src/**/* include.
- src/config.ts exporting a config object with database, api, and aws credentials.
- src/database.ts with getUserByEmail, searchProducts, deleteUser, and updateUserRole functions using string interpolation.
- src/auth.ts with registerUser, loginUser, validateSession, and isAdmin functions storing plaintext passwords.
- src/server.ts with parseBody, POST routes for /register and /login, GET routes for /user and /search, and server.listen(3000).
Verify the project compiles
The code is deliberately insecure, but it should still be valid TypeScript. If tsc reports type errors, something was pasted incorrectly.
- Before you run the type checker, predict: do you expect any errors from this code?
- Run the TypeScript compiler to check for type errors:
npx tsc --noEmitThe command produces no output and returns you to the prompt. That silence means zero type errors.
Why no output?
The --noEmit flag tells TypeScript to check types without generating JavaScript files. When there are no errors, it exits silently with code 0. This is exactly what you want: proof that your vulnerable code is syntactically valid.
- Confirm your commit history by running:
git log --onelineYou should see two commits listed, most recent first:
abc1234 add vulnerable source files
def5678 initial project setupWhat should I see?
Your commit hashes will be different (they are unique to your repository). The important thing is seeing both commit messages confirming your project scaffold and source files are tracked.
Seeing type errors from tsc?
If npx tsc --noEmit reports errors, the most common causes are:
- A file was not saved after pasting. Re-open each src/ file and save it.
- Content was pasted out of order in database.ts, auth.ts, or server.ts. Compare your file against the chunks above. Each chunk goes directly below the previous one.
- The tsconfig.json has a typo. Make sure rootDir points to ./src.
help me fix TypeScript compilation errors in my secure-review-lab project
Your vulnerable testing ground is ready. Next up, you will teach Claude Code your security standards and build the skill that reviews this code for you.
Create CLAUDE.md and Your First Security Review Skill
Your vulnerable TypeScript app is committed and compiles cleanly. The next problem: Claude Code does not know what your team considers a security issue unless you tell it explicitly.
Claude Code reads a CLAUDE.md file at the start of every session to understand your project's standards. You'll create one that defines five security categories and severity levels. Then you'll build a custom skill that uses dynamic context injection to pull your live git diff into the prompt so Claude reviews actual code changes.
In this step, get ready to:
- Create a CLAUDE.md file that defines your security standards and severity levels.
- Build a custom /security-review skill with dynamic context injection.
- Test the skill on an uncommitted code change.
Create CLAUDE.md with your security standards
Claude Code reads CLAUDE.md at the start of every session. It is your project's persistent memory. Anything you put here shapes how Claude responds for the entire session.
- Create a file called CLAUDE.md at the root of your secure-review-lab project by running:
touch CLAUDE.md- Open CLAUDE.md in your editor and paste the project header, build info, and security standards:
# Secure Review Lab
TypeScript/Node.js project for building and testing a security review skill.
## Build
- Typecheck: `npx tsc --noEmit`
- No runtime execution needed (sample app is for static analysis only)
## Security Standards
This project enforces five security review categories:
1. **Secrets**: No hardcoded API keys, passwords, tokens, or credentials in source code
2. **Injection**: No SQL/command/code injection via string concatenation or template literals with user input
3. **Validation**: All user input must be validated (type, format, length) before use
4. **Authentication**: Passwords must be hashed, sessions must expire, tokens must be cryptographically random
5. **Unsafe Patterns**: No eval(), Function constructor, dynamic require(), or unsanitized exec() with user inputWhat do these sections do?
The Build section tells Claude how to check your code (typecheck only, no runtime needed). The Security Standards section defines exactly which vulnerability categories to look for. Claude now knows your team's rules without you repeating them each session.
- Below the Security Standards section in the same file, paste the severity levels and conventions:
## Severity Levels
- Critical: Exploitable immediately with no authentication (e.g., exposed secrets, unauthenticated SQL injection)
- High: Exploitable with minimal effort (e.g., weak auth, command injection behind login)
- Medium: Requires specific conditions to exploit (e.g., missing validation that could lead to injection)
- Low: Defense-in-depth issues (e.g., missing rate limiting, verbose errors)
## Conventions
- All source code in `src/`
- TypeScript strict mode enabled
- Node.js built-in modules only (no external runtime dependencies)Why define severity levels here?
Without explicit severity definitions, Claude might label everything as "High" or use inconsistent scales across sessions. By defining Critical through Low with concrete examples, you get consistent prioritization every time the skill runs.
- Save CLAUDE.md.
- Confirm the file exists by running:
ls CLAUDE.mdYou should see CLAUDE.md listed in the output.
Don't see CLAUDE.md?
Make sure you are in the secure-review-lab directory. Run pwd to check your current location. If you are somewhere else, run cd ~/Desktop/secure-review-lab to get back.
Help me find my file
✔️ Awesome, I've got everything!
Great. Make sure CLAUDE.md is saved with both sections (Security Standards and Severity Levels + Conventions).
ⓧ I'd like to double check the full code
Your CLAUDE.md should contain two sections in order. Scroll up and confirm:
- The first block starts with # Secure Review Lab and ends with the five numbered Security Standards categories.
- The second block starts with ## Severity Levels and ends with the three Conventions bullet points.
Create the security review skill
A skill is a reusable command you invoke with a slash. The directory name becomes the command. Inside, a SKILL.md file holds YAML frontmatter (tells Claude when to use it) and markdown instructions (tells Claude what to do).
- Create the skill directory structure by running:
mkdir -p .claude/skills/security-review- Confirm the directory was created by running:
ls .claude/skills/security-reviewThe directory should exist. The output may be empty since there are no files in it yet.
- Create the skill file and open it in your editor by running:
touch .claude/skills/security-review/SKILL.md- Open .claude/skills/security-review/SKILL.md in your editor and paste the following content:
---
description: Reviews uncommitted code changes for security vulnerabilities. Use when the user asks to check security, review changes for risks, or before committing code. Inspects for secrets, injection, validation gaps, auth weaknesses, and unsafe patterns.
---
## Changes to review
!`git diff HEAD`
## Instructions
Analyze the code changes above for security vulnerabilities. Follow the format shown in [examples/sample-report.md](examples/sample-report.md) exactly.
Check all five categories:
1. **Secrets** - Hardcoded API keys, passwords, tokens, database credentials, or private keys
2. **Injection** - SQL injection, command injection, or code injection via string concatenation or template literals with user input
3. **Validation** - Missing input validation (type checks, format validation, length limits, sanitization)
4. **Authentication** - Plaintext passwords, predictable tokens, missing session expiration, weak comparisons
5. **Unsafe Patterns** - eval(), Function constructor, dynamic require(), child_process.exec() with user input
For each finding:
- Assign severity: Critical > High > Medium > Low
- Order findings by severity (Critical first)
- Include the affected file path and relevant code snippet
- Explain WHY it is a vulnerability (attack scenario)
- Provide a concrete fix with code
If a category has zero findings, state: "No [category] issues found."
If the diff is empty, say: "No uncommitted changes to review."What does this skill do?
The description in the YAML frontmatter tells Claude when to load this skill automatically. If you ask anything about security or reviewing changes, Claude knows to reach for this skill.
The line !`git diff HEAD` is dynamic context injection. Before Claude ever sees the skill content, Claude Code runs that command on your machine and replaces the line with its output. Claude receives the actual diff already inlined, so it analyzes real code changes.
- Save SKILL.md.
- Verify the skill file is in place by running:
ls .claude/skills/security-review/SKILL.mdYou should see the file path printed back to you.
File not found?
Double check the directory structure. Run find .claude -name SKILL.md to see where it ended up. The path must be exactly .claude/skills/security-review/SKILL.md for Claude Code to recognize it as the /security-review command.
Help me fix my skill directory structure
✔️ Awesome, I've got everything!
Great. Make sure both CLAUDE.md and .claude/skills/security-review/SKILL.md are saved.
ⓧ I'd like to double check the full code
Here is the full SKILL.md you should have:
---
description: Reviews uncommitted code changes for security vulnerabilities. Use when the user asks to check security, review changes for risks, or before committing code. Inspects for secrets, injection, validation gaps, auth weaknesses, and unsafe patterns.
---
## Changes to review
!`git diff HEAD`
## Instructions
Analyze the code changes above for security vulnerabilities. Follow the format shown in [examples/sample-report.md](examples/sample-report.md) exactly.
Check all five categories:
1. **Secrets** - Hardcoded API keys, passwords, tokens, database credentials, or private keys
2. **Injection** - SQL injection, command injection, or code injection via string concatenation or template literals with user input
3. **Validation** - Missing input validation (type checks, format validation, length limits, sanitization)
4. **Authentication** - Plaintext passwords, predictable tokens, missing session expiration, weak comparisons
5. **Unsafe Patterns** - eval(), Function constructor, dynamic require(), child_process.exec() with user input
For each finding:
- Assign severity: Critical > High > Medium > Low
- Order findings by severity (Critical first)
- Include the affected file path and relevant code snippet
- Explain WHY it is a vulnerability (attack scenario)
- Provide a concrete fix with code
If a category has zero findings, state: "No [category] issues found."
If the diff is empty, say: "No uncommitted changes to review."For CLAUDE.md, scroll up and confirm it contains both sections: the header through Security Standards, and the Severity Levels through Conventions.
Test the skill in Claude Code
The skill is in place. Time to see it work on a real code change.
- Start Claude Code in the project directory by running:
claudeWhat happens when Claude Code starts?
Claude Code reads your CLAUDE.md and discovers the /security-review skill in .claude/skills/. It now has your security standards in memory and knows the skill exists. You may need to grant file read permissions the first time it accesses files.
Before you can test the skill, you need an uncommitted change for it to find.
- Open a second terminal tab (or exit Claude Code temporarily with Ctrl+C).
- Add a small insecure line to any file. For example, append this to src/config.ts:
echo '
export const testSecret = "ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";' >> src/config.ts- Do NOT commit this change. Leave it as an uncommitted edit.
- Confirm the change exists in your working tree by running:
git diff HEAD --statYou should see src/config.ts listed as modified.
- Return to Claude Code (run claude again if you exited, or switch to the tab where it is running).
Before you invoke the skill, what do you think it will find? You just added a hardcoded GitHub token. Will Claude catch it and label it Critical?
- Invoke the skill by typing this in the Claude Code prompt:
/security-reviewClaude runs git diff HEAD behind the scenes, injects the diff output into the skill, and responds with a security report. You should see at least one finding about the hardcoded token you just added, categorized under Secrets with a Critical severity.
Why dynamic context injection matters
Without !`git diff HEAD`, Claude would have to guess which files changed or read every file in the project. Dynamic context injection runs the command first and hands Claude the exact diff. The skill always operates on real, current changes.
Skill not found or no output?
- Make sure the directory name is exactly security-review (not security_review or securityreview).
- Verify the file is named SKILL.md (capital letters matter).
- If you started Claude Code before creating the skill, exit and restart it. Claude Code discovers skills at session start.
Help me debug my skill
- After confirming the skill works, revert your test change so the working tree is clean for the next step:
git checkout -- src/config.tsYour security review skill is live. You can type /security-review any time you have uncommitted changes and get a prioritized vulnerability report. Next up, you'll push this skill to its limits and discover why the output needs more structure.
Hit the Wall with Inconsistent Results
Your /security-review skill works. It scans your diff, finds vulnerabilities, and returns findings. That sounds like a success.
But "works" and "works reliably" are different things. In this step, you will push the skill harder with a new file full of unsafe patterns and discover that the output is inconsistent across runs. The format changes, findings get missed, severity ordering is wrong. This inconsistency is the problem you will solve in the next step.
In this step, get ready to:
- Add a new file with four different unsafe code patterns to stress-test the skill.
- Observe inconsistent skill output across multiple runs of the same diff.
- Use Plan Mode to diagnose why the output is unreliable without editing any files.
Add a new vulnerable file to stress-test the skill
Your existing files cover secrets, injection, validation, and auth. You need a file that focuses on a different class of vulnerability: unsafe runtime patterns like eval(), shell injection, and dynamic code loading.
- Create a new file at src/api.ts and add the first two unsafe functions by pasting this code:
// DELIBERATELY VULNERABLE - for security review testing
// Added in Step 3 to test additional vulnerability patterns
import { exec } from "node:child_process";
export function processTemplate(userInput: string): unknown {
// UNSAFE PATTERN: eval with user input
return eval(userInput);
}
export function runDiagnostic(command: string): void {
// UNSAFE PATTERN: shell injection via exec with user input
exec(`diagnostic-tool --check ${command}`, (error, stdout) => {
if (error) {
console.error(`Diagnostic failed: ${error.message}`);
return;
}
console.log(stdout);
});
}What does this code do?
These two functions are deliberately dangerous.
- processTemplate runs eval() directly on user input. An attacker can execute any JavaScript on your server.
- runDiagnostic passes user input into child_process.exec() without sanitization. An attacker can inject shell commands.
- Add the remaining two unsafe functions below the runDiagnostic function in the same file:
export function loadPlugin(pluginName: string): unknown {
// UNSAFE PATTERN: dynamic require with user-controlled path
const modulePath = `../plugins/${pluginName}`;
return require(modulePath);
}
export function renderContent(template: string, data: Record<string, string>): string {
// UNSAFE PATTERN: Function constructor (equivalent to eval)
const fn = new Function("data", `return \`${template}\``);
return fn(data) as string;
}What does this code do?
- loadPlugin uses a dynamic require() with a user-controlled path. An attacker can load arbitrary modules from your filesystem.
- renderContent creates a new Function() from a user-supplied template string. This is functionally equivalent to eval().
- Save src/api.ts.
- Confirm the file compiles cleanly by running:
npx tsc --noEmitYou should see no errors. Do NOT commit this file yet. You want it to appear in the uncommitted diff so the skill can find it.
Seeing type errors?
- Check that the import statement at the top says import { exec } from "node:child_process" with the node: prefix.
- Confirm the file is saved inside the src/ directory, not the project root.
Help me debug my TypeScript compilation error in src/api.ts
✔️ Awesome, I've got everything!
Great. Double check you have saved src/api.ts and confirmed no type errors.
ⓧ I'd like to double check the full code
Here is the complete src/api.ts file. Confirm your version matches:
// DELIBERATELY VULNERABLE - for security review testing
// Added in Step 3 to test additional vulnerability patterns
import { exec } from "node:child_process";
export function processTemplate(userInput: string): unknown {
// UNSAFE PATTERN: eval with user input
return eval(userInput);
}
export function runDiagnostic(command: string): void {
// UNSAFE PATTERN: shell injection via exec with user input
exec(`diagnostic-tool --check ${command}`, (error, stdout) => {
if (error) {
console.error(`Diagnostic failed: ${error.message}`);
return;
}
console.log(stdout);
});
}export function loadPlugin(pluginName: string): unknown {
// UNSAFE PATTERN: dynamic require with user-controlled path
const modulePath = `../plugins/${pluginName}`;
return require(modulePath);
}
export function renderContent(template: string, data: Record<string, string>): string {
// UNSAFE PATTERN: Function constructor (equivalent to eval)
const fn = new Function("data", `return \`${template}\``);
return fn(data) as string;
}Run the skill multiple times and observe the inconsistency
Your diff now includes four new unsafe functions on top of the existing committed vulnerabilities. The skill should have plenty to find.
- Switch back to your Claude Code session in your terminal.
- Run the security review skill by typing:
/security-reviewYou will see security findings. Read through the output and note the format: how are findings organized? What severity labels does it use? What order are they in?
Before you run the skill again, predict: do you expect the exact same output, word for word, in the same format and order?
- Run /security-review a second time on the same unchanged diff.
- Run /security-review a third time.
- Compare all three outputs side by side.
The outputs are different each time. The same diff, the same skill, three different results.
What kind of inconsistencies will you see?
Look for these patterns across your three runs:
- Findings appear in a different order each time. One run starts with secrets, another with injection.
- Some runs miss the eval() issue or the Function constructor entirely.
- Severity labels change between runs. The same vulnerability might be "Critical" in one and "High" in another.
- The format shifts between bullets, tables, and prose paragraphs with no consistency.
Why is this happening?
The SKILL.md tells Claude to follow the format in examples/sample-report.md, but that file does not exist yet. Without a concrete template to follow, Claude improvises the format differently each time.
Use Plan Mode to diagnose the root cause
You could start editing the skill file right now to fix this. But guessing at a fix risks making things worse. Plan Mode lets Claude analyze the problem and propose a structured plan without touching any files.
- Toggle into Plan Mode by pressing Shift+Tab until the status bar shows "plan" mode.
What is Plan Mode?
Pressing Shift+Tab cycles through three permission modes: default > acceptEdits > plan. In plan mode, Claude reads files and proposes changes but makes no edits until you approve them. This is ideal for diagnosing problems without accidentally breaking things.
- Ask Claude to analyze the skill's inconsistency by typing this prompt:
Read the /security-review skill at .claude/skills/security-review/SKILL.md and analyze why it produces inconsistent output format and missed findings across multiple runs. What structural changes would make the output reliable and consistent?Claude reads the skill file, identifies the structural weaknesses, and proposes a plan. It will not edit anything because you are in Plan Mode.
- Read through Claude's plan. Look for these key suggestions:
- The missing examples/sample-report.md file needs to be created with a concrete output template.
- The skill instructions need stronger enforcement of severity ordering (Critical first).
- Each category should be explicitly checked and reported, even when no findings exist.
Save Claude's plan for the next step
You do not need to copy the plan anywhere. Claude retains the conversation context. In the next step, you will act on these suggestions by creating the template file and updating the skill.
- Toggle back to default mode by pressing Shift+Tab until the status bar shows "default".
Plan Mode not toggling?
- Make sure the input line is empty before pressing Shift+Tab. If you have text in the input, the shortcut may not register.
- Alternatively, start a fresh session with claude --permission-mode plan to force plan mode from the start.
Help me toggle into Plan Mode in Claude Code
You now have a clear diagnosis: the skill lacks a concrete output template, and its instructions are not specific enough to force consistent results. In the next step, you will create that template, update the skill, and use checkpoints to iterate safely until the output is rock-solid.
Fix the Skill with Structured Output and Checkpoints
In the previous step, Plan Mode revealed why your skill's output kept changing between runs. Without a concrete example of the expected format, Claude improvises the structure each time.
The fix is straightforward. You will create an example report that shows Claude exactly what the output should look like. Combined with checkpoints, you can safely experiment with the template and rewind if an iteration makes things worse.
In this step, get ready to:
- Create a structured output template that Claude follows on every run.
- Use checkpoints to safely iterate on the skill.
- Verify three consecutive runs produce identical formatting.
Create the example report template
Skills can include supporting files like templates and examples in the skill directory. Your SKILL.md already references examples/sample-report.md, but that file does not exist yet. Creating it gives Claude a concrete template to follow.
- First, commit src/api.ts so it becomes part of your project history. Run these commands:
git add src/api.ts
git commit -m "Add api.ts with unsafe patterns for testing"- Create the examples directory inside your skill folder by running:
mkdir -p .claude/skills/security-review/examples- Confirm the directory was created by running:
ls .claude/skills/security-review/You should see both SKILL.md and the new examples/ directory listed.
Now create the example report file. This template shows the exact structure Claude should produce: a summary header with counts, findings grouped by severity, each with a category tag, file path, code snippet, risk explanation, and fix.
- Create .claude/skills/security-review/examples/sample-report.md and add the report header and first Critical finding:
# Security Review Report
**Summary**: 3 Critical, 1 High, 2 Medium, 0 Low
---
## Critical
### [Secrets] Hardcoded database password in src/config.ts
```typescript
password: "SuperSecret123!",
```
**Risk**: Credentials committed to source control can be extracted by anyone with repo access and used to access the production database directly.
**Fix**: Use environment variables:
```typescript
password: process.env.DB_PASSWORD ?? "",
```
---What does this structure show Claude?
The header line **Summary**: 3 Critical, 1 High, 2 Medium, 0 Low teaches Claude to always open with a count. Each finding follows the same shape: category tag in brackets, affected file in the heading, a code snippet, a risk explanation, and a fix with code.
- Add the second Critical finding and the High severity section below the --- separator:
### [Injection] SQL injection in src/database.ts
```typescript
const query = `SELECT * FROM users WHERE email = '${email}'`;
```
**Risk**: An attacker can pass `' OR '1'='1` as the email to dump all users, or use UNION-based injection to extract other tables.
**Fix**: Use parameterized queries:
```typescript
const query = { text: "SELECT * FROM users WHERE email = $1", values: [email] };
```
---
## High
### [Unsafe Patterns] eval() with user input in src/api.ts
```typescript
return eval(userInput);
```
**Risk**: Arbitrary code execution. An attacker can run any JavaScript on the server by passing malicious input.
**Fix**: Use a safe parser or explicit mapping:
```typescript
return JSON.parse(userInput);
```- Add the Medium severity section and the No Issues section to complete the template:
---
## Medium
### [Validation] No input length limit in src/server.ts
```typescript
const body = await parseBody(req);
```
**Risk**: An attacker can send an extremely large request body to exhaust server memory (denial of service).
**Fix**: Add a size limit:
```typescript
const MAX_BODY = 1024 * 100; // 100KB
if (body.length > MAX_BODY) { res.writeHead(413); res.end(); return; }
```
---
## No Issues
- No **Authentication** issues found in this diff.Why include a No Issues section?
The ## No Issues section teaches Claude to explicitly state when a category has zero findings. Without it, Claude might silently skip categories, and you would not know whether it checked or just forgot.
- Save the file.
✔️ Awesome, I've got everything!
Great. Double check you have saved sample-report.md with all three severity sections (Critical, High, Medium) plus the No Issues footer.
ⓧ I'd like to double check the full code
Your sample-report.md should contain all these sections in order. Scroll up and confirm each chunk is present:
- Report header with **Summary** counts line and a ## Critical heading.
- First finding: [Secrets] Hardcoded database password in src/config.ts with Risk and Fix.
- Second finding: [Injection] SQL injection in src/database.ts with Risk and Fix.
- High section: [Unsafe Patterns] eval() with user input in src/api.ts with Risk and Fix.
- Medium section: [Validation] No input length limit in src/server.ts with Risk and Fix.
- No Issues section stating no Authentication issues found.
Confirm the skill references the template
Your SKILL.md already contains the instructions that enforce consistent output. Before testing, confirm the reference is in place and understand how checkpoints protect your work.
- Open .claude/skills/security-review/SKILL.md and find this line:
Analyze the code changes above for security vulnerabilities. Follow the format shown in [examples/sample-report.md](examples/sample-report.md) exactly.This line points Claude at the example file you just created. The rest of the Instructions section enforces severity ordering and the five-category check.
How do checkpoints protect you here?
Every prompt you send in Claude Code automatically creates a checkpoint. If your next edit to SKILL.md makes output worse, you can run /rewind to open the rewind menu and select the point before your edit.
Choose Restore code and conversation to get back to a known-good state. You can also press Esc twice when the prompt input is empty to open the same menu. Checkpoints only track files edited through Claude Code's editing tools, not files changed by bash commands.
Test the skill, iterate, and commit
With the template in place, it is time to see if the skill produces consistent output. You will run it multiple times on the same codebase and compare the results.
- Return to your Claude Code session.
- Before you run the skill, think about what you expect. The diff now includes all five source files. Do you expect the report to cover all five vulnerability categories in a consistent format?
- Invoke the skill by typing:
/security-reviewClaude should respond with a structured report following the template: a summary line with severity counts, then findings grouped by severity (Critical first), each with a category tag, code snippet, risk explanation, and fix.
- Run /security-review two more times (three runs total). Compare the outputs.
What to look for across runs
All three runs should share the same structure: same heading format, same severity ordering (Critical first, then High, then Medium), and the same category tags in brackets. The exact wording of risk explanations may vary slightly, but the structure should be locked.
Output still looks inconsistent?
If the format varies between runs, the template may need stronger anchoring. Try this iteration loop:
- Run /rewind and select the checkpoint before your last SKILL.md or template edit.
- Choose Restore code and conversation to revert.
- Try adding more explicit phrasing to SKILL.md, such as "Copy the exact heading format from the example" or "Use the identical markdown structure".
- Test again with three consecutive runs.
Help me diagnose my inconsistent skill output
- Once three runs produce the same structured format, commit all skill files by running:
git add .claude/
git commit -m "Add structured output template for security-review skill"- Verify the commit by running:
git log --oneline -3You should see your latest commit for the skill template, the api.ts commit, and your earlier project commits listed.
Your skill now produces reliable, structured output every time. Next up, you will build an automated test script that proves the skill catches all five vulnerability categories without manual checking.
Add an Automated Test Script
Your skill now produces consistent, structured output every time. But testing it manually every time you tweak the SKILL.md is tedious and error-prone.
An automated test script solves this. It plants one known vulnerability per category, runs the skill non-interactively, and checks that every category appears in the output. If you ever change the skill instructions and accidentally break detection for a category, the script catches it instantly.
In this step, get ready to:
- Write a bash test script that plants one vulnerability per category in a temporary branch.
- Add assertion logic that greps the skill output for expected keywords.
- Run the test and confirm all five categories pass.
Create the test script structure
The test script needs to isolate its work on a temporary git branch so it does not pollute your main codebase. It creates a file with one clear vulnerability per category, then invokes the skill to analyze it.
- Create the tests/ directory by running:
mkdir tests- Confirm the directory exists by running:
lsYou should see tests listed alongside src and your other project files.
- Create tests/security-review-test.sh and add the script header and setup section:
#!/bin/bash
# Automated test for the /security-review skill
# Plants known vulnerabilities and verifies the skill catches them
set -e
PASS=0
FAIL=0
TOTAL=5
echo "=== Security Review Skill Test ==="
echo ""
# Save current branch
ORIGINAL_BRANCH=$(git branch --show-current)
# Create a test branch
git checkout -b test-security-review-$$ 2>/dev/nullWhat does this code do?
- set -e makes the script exit immediately if any command fails, preventing silent errors.
- PASS, FAIL, and TOTAL track the test results as the script runs.
- $$ is the current process ID. It makes the branch name unique so parallel runs do not collide.
- ORIGINAL_BRANCH saves where you are so the script can switch back after testing.
- Below the branch creation line, add the heredoc that plants one vulnerability per category into a test file:
# Create a test file with one example of each vulnerability category
cat > src/test-vulnerable.ts << 'EOF'
// Test file: one vulnerability per category
// SECRETS: hardcoded token
const API_TOKEN = "ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
// INJECTION: SQL via concatenation
function findUser(id: string) {
return `DELETE FROM users WHERE id = ${id}`;
}
// VALIDATION: no checks on input
function processAge(age: any) {
return age * 2;
}
// AUTH: plaintext password storage
function storeUser(email: string, password: string) {
const record = { email, password, created: Date.now() };
return record;
}
// UNSAFE: eval with input
function compute(expression: string) {
return eval(expression);
}
export { API_TOKEN, findUser, processAge, storeUser, compute };
EOFWhat does this heredoc do?
The cat > file << 'EOF' syntax writes everything between << and EOF into the file. Each function contains exactly one vulnerability pattern so the test can verify each category independently.
The single quotes around 'EOF' prevent bash from expanding the ${id} template literal inside the TypeScript code.
Add assertion checks and cleanup logic
The script now plants vulnerabilities, but it also needs to invoke the skill, check the output, and clean up after itself.
- Below the EOF line, add the skill invocation and the check_category function with its five calls:
git add src/test-vulnerable.ts
echo "Running /security-review on test vulnerabilities..."
echo ""
# Run the skill in non-interactive mode and capture output
OUTPUT=$(claude -p "/security-review" 2>/dev/null || true)
# Check each category
check_category() {
local category="$1"
local pattern="$2"
if echo "$OUTPUT" | grep -qi "$pattern"; then
echo " PASS: $category detected"
PASS=$((PASS + 1))
else
echo " FAIL: $category NOT detected"
FAIL=$((FAIL + 1))
fi
}
check_category "Secrets" "secret\|token\|credential\|API_TOKEN\|hardcoded"
check_category "Injection" "injection\|SQL\|concatenat"
check_category "Validation" "validation\|sanitiz\|no.*check\|unvalidated\|missing.*input"
check_category "Authentication" "plaintext\|password.*stor\|hash\|bcrypt"
check_category "Unsafe Patterns" "eval\|unsafe\|arbitrary.*code\|code.*execution"What does this code do?
- git add stages the test file so it appears in git diff HEAD (which the skill uses for dynamic context injection).
- claude -p "/security-review" runs Claude Code in non-interactive mode. The -p flag sends the prompt and exits immediately.
- || true prevents the script from aborting if Claude exits with a non-zero code.
- check_category greps the captured output for patterns that indicate a category was detected. Each call uses multiple alternatives separated by \| to match different phrasings Claude might use.
- Finally, add the results summary, cleanup, and exit logic below the five check_category calls:
echo ""
echo "=== Results: $PASS/$TOTAL passed, $FAIL failed ==="
# Cleanup: remove test file and branch
git checkout -- src/test-vulnerable.ts 2>/dev/null || true
git reset HEAD src/test-vulnerable.ts 2>/dev/null || true
rm -f src/test-vulnerable.ts
git checkout "$ORIGINAL_BRANCH" 2>/dev/null
git branch -D "test-security-review-$$" 2>/dev/null
if [ $FAIL -eq 0 ]; then
echo "All categories detected! Skill is working correctly."
exit 0
else
echo "Some categories were missed. Review SKILL.md instructions."
exit 1
fiWhat does the cleanup do?
The cleanup section undoes everything the script created: it unstages and removes the test file, switches back to your original branch, and deletes the temporary branch. This means you can run the test repeatedly without leftover state.
The exit code (0 for success, 1 for failure) makes the script usable in CI pipelines or pre-commit hooks later.
- Save tests/security-review-test.sh.
✔️ Awesome, I've got everything!
Great. Double check you have saved the file before continuing.
ⓧ I'd like to double check the full code
Scroll up and confirm your tests/security-review-test.sh contains these sections in order:
- The shebang, set -e, counter variables, and branch creation (chunk 1).
- The heredoc that creates src/test-vulnerable.ts with five vulnerabilities (chunk 2).
- The git add, OUTPUT capture, check_category function, and five category calls (chunk 3).
- The results summary, cleanup commands, and exit logic (chunk 4).
File looks wrong or has syntax errors?
- Make sure the EOF closing the heredoc sits alone on its own line with no leading whitespace.
- Check that you have matching quotes on every echo and check_category line. A missing closing quote will break the entire script.
- Verify the backslash-pipe patterns (\|) in the grep patterns have no spaces around them.
Help me debug my security-review-test.sh script
Run the test script
The script is ready. Time to make it executable and run it to see if your skill catches all five vulnerability categories.
- Make the script executable by running:
chmod +x tests/security-review-test.shBefore you run the test, think about this: do you expect all five categories to pass on the first run, or will some be missed?
- Run the test script by executing:
bash tests/security-review-test.shYou should see output showing PASS for each of the five categories, followed by a summary line reading 5/5 passed, 0 failed and the message All categories detected! Skill is working correctly..
Seeing FAIL for one or more categories?
This is expected if the skill's output uses different phrasing than what the grep patterns look for. Here is how to fix it:
- Run claude -p "/security-review" manually to see the full output. Find the phrasing used for the failing category.
- Update the corresponding check_category line in the script to include a pattern that matches the actual phrasing.
- Re-run bash tests/security-review-test.sh until all five pass.
My security review test script is failing for some categories
- Once all five categories pass, commit the test script by running:
git add tests/security-review-test.sh
git commit -m "Add automated test script for security review skill"- Verify the commit by running:
git log --oneline -1You should see your new commit message at the top.
Why non-interactive testing matters
The -p flag lets you run Claude Code skills without an interactive session. This means you can trigger them from scripts, CI pipelines, or git hooks. Your test script proves the skill works reliably without a human watching.
If you ever update your SKILL.md instructions, re-running this script immediately tells you whether you accidentally broke detection for any category.
Your security review skill is now backed by an automated test suite. Any future changes to the skill can be validated in seconds. Next up, you will tackle the Secret Mission and extend the skill to support staged-only and branch-specific review modes.
Secret mission
Review Only Staged or Branch-Specific Changes
Your skill reviews all uncommitted changes, but real workflows need precision. What if you could review only what you are about to commit, or only what a feature branch introduced? Wire up argument handling to make your skill practical for pre-commit hooks and PR reviews.
Clean Up Your Resources
Clean Up Your Resources
Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.
Resources you used:
- The secure-review-lab/ project directory (source files, .claude/ skills directory, tests, and node_modules/).
Keep everything running
No action needed. Choose this if you plan to keep developing your security review skill or want to use it in future projects.
- Your /security-review skill lives in .claude/skills/security-review/ inside this project.
- To make the skill available across all your projects, copy it to ~/.claude/skills/security-review/.
- To share the skill with teammates, commit the .claude/ directory to version control so they get the skill automatically.
Pause - I'll come back to this later
There are no running processes or servers to stop. The project is already paused by default when you close Claude Code.
- Exit Claude Code by typing /exit or pressing Ctrl+C.
- Your project files and skill remain in the secure-review-lab/ directory on your Desktop, ready for you to return.
Delete - I don't want to use this again
Remove all project files. No other cleanup is needed since everything is local.
- Exit Claude Code by typing /exit or pressing Ctrl+C.
- Delete the project directory by running:
cd ~/Desktop
rm -rf secure-review-labOn Windows, use:
cd ~\Desktop
rmdir /s /q secure-review-lab- Confirm the folder is gone by running ls ~/Desktop and verifying secure-review-lab no longer appears.
Nice Work!
Nice Work!
You just built a custom security review skill for Claude Code from scratch. You can now type /security-review in any project and get a structured, prioritized report of security vulnerabilities in your code changes.
Here's what you learned:
- Built a custom /security-review skill using dynamic context injection to pull live git diff output into the prompt, giving Claude real code changes to analyze instead of guessing from open files.
- Used Plan Mode to diagnose inconsistent skill output without making edits, then used checkpoints and /rewind to safely iterate on a structured output template until results were consistent across runs.
- Created an automated test script that plants known vulnerabilities and verifies the skill catches all five categories, proving reliability and catching regressions.
- Extended the skill to support staged-only and branch-specific review modes for real-world pre-commit and PR review workflows.
Ready to quiz yourself?