Test AI Prompts with n8n

Build an n8n workflow that tests AI prompts and tracks pass rates.

Introduction

30 Second Summary

A single good answer can make an edited instruction feel ready to use. The failures you did not retry can still slip through unnoticed.

In this project, you will build an n8n workflow that tests one prompt against a fixed list of inputs. It checks each result against an expected outcome before showing which cases passed or failed.

What You'll Build

You run your workflow to see a table of pass-or-fail results, improve the prompt using the named failures, then rerun the same cases to watch the pass count change.

By the end of this project, you'll have:

  • A repeatable prompt test that sends at least eight fixed inputs through the same AI node.
  • A pass-or-fail checker that compares every answer with its expected outcome and exposes the first broken cases.
  • A side-by-side run history that shows how two prompt versions performed against the same test list.
  • Secret Mission: An optional challenge to push your skills further.

Are there any prerequisites?

You need access to n8n through an account or a self-hosted instance. You also need one model API key plus at least eight inputs with known expected answers.

Expect the short model calls in this project to cost between $0 and about $1.

Before We Start

A repeatable prompt test needs a reliable place to run the same cases. n8n gives you a visual editor for building that test harness.

Calls to OpenAI require secure authentication. n8n keeps the credential separate from the workflow that uses it.

In this step, you prepare both services before creating any test cases. You finish inside the blank workflow editor.

In this step, get ready to:
  • Sign in to an n8n workspace.
  • Store an OpenAI API key in n8n.
  • Reach the blank workflow editor.
Enter your n8n workspace

n8n Cloud provides a managed workspace without a local installation. Its free trial gives you a quick route into the editor.

  • Open the n8n Cloud registration page in your browser.
  • Complete the account prompts to create your workspace.
  • Complete any email verification requested by n8n.
  • Finish the onboarding prompts shown by n8n.

You should reach the Overview page inside your workspace. Your workspace is ready to hold the prompt test harness.

Unable to Reach Your Workspace?

  • Check your inbox for the verification message from n8n.
  • Check your spam folder if the message is missing.
  • Return to the registration page if the verification link has expired.

Still blocked? Help me troubleshoot my n8n workspace access. You can also search the n8n community forum for account setup help.

Create the OpenAI credential

An API credential lets n8n authenticate when an AI node sends a model request. Saving it once also lets later nodes use it without exposing the key inside the workflow.

Keep Your API Key Private

This next action handles a live credential, so a little care matters. The key can authorize API usage on your account.

  • Paste the key only into the n8n API Key field.
  • Keep the key out of screenshots.
  • Keep the key out of workflow fields.
  • Open the OpenAI API keys page in a new browser tab.
  • Create an OpenAI account if you do not already have one.
  • Sign in to your OpenAI account.
  • Select Create new secret key.
  • Copy the key when OpenAI displays it.

That is the sensitive part handled. Your key is ready to move into n8n without placing it in the workflow.

  • Switch back to your n8n workspace tab.
  • Select Create in the upper-left side menu.
  • Select Credential.
  • Choose your personal space if n8n asks for a location.
  • Select OpenAI as the service.
  • Select the current credential name at the top of the credential window.
  • Replace the current name with OpenAI.
  • Paste the secret key into the API Key field.

When Is an Organization ID Needed?

The Organization ID field is required when your OpenAI account belongs to multiple organizations. A single-organization account can leave this field empty.

OpenAI provides the value in Organization Settings. Requests that include the value count toward that organization's subscription quota.

  • Leave the Organization ID field empty if your account belongs to one organization.
  • Enter your Organization ID if your account belongs to multiple organizations.

Before you save, do you expect n8n to accept the credential or report a connection problem?

  • Select Save to store the credential.

n8n tests the credential when it saves. You should see the credential remain available without a failed connection notice.

Credential Test Failed?

  • Paste the entire secret key into the API Key field again.
  • Remove any spaces copied before the key.
  • Create a replacement key from the OpenAI API keys page if the original key is unavailable.

Need help with the visible status? Help me diagnose the n8n credential test safely. You can also search the n8n community forum for credential setup help.

Credential Checkpoint

The credential named OpenAI is configured and available for use by n8n AI nodes. Its API key value is stored securely in n8n.

The API key value is not exposed in the workflow.

Open the blank workflow editor

The workflow editor contains the canvas where you connect nodes. An empty canvas gives the next step a clean starting point.

Before you open the editor, do you expect to see a node or an empty canvas?

  • Return to the Overview page in your n8n workspace.
  • Select Create Workflow.
  • Choose your personal space if n8n asks for a location.

You should see Add first step... on a blank canvas. The canvas should contain no nodes.

No test cases or execution results exist yet. The run-history table does not exist yet.

Editor Did Not Open?

  • Return to the Overview page.
  • Find the workflows list on the page.
  • Select Create Workflow again.

Still seeing a different page? Help me reach the blank n8n workflow editor. You can also search the n8n community forum for editor navigation help.

✔️ Awesome, I've got everything!

Your workspace is open. Your OpenAI credential is saved securely. Your blank workflow canvas is ready for the first node.

ⓧ I'd like to double check the setup

  • Confirm that you are signed in to an n8n workspace.
  • Confirm that the credential is named OpenAI.
  • Confirm that the credential is available to n8n AI nodes.
  • Confirm that the API key value is hidden from the workflow.
  • Confirm that the workflow editor shows a blank canvas.

Your workspace now has secure access to OpenAI. Next, you build one AI node and inspect its first answer in the execution view.

Run Your First Prompt

One successful reply proves only that the path can run. Before you compare prompt versions, you need a baseline execution that you can inspect.

This step creates the smallest working path in n8n. You will send one customer message through prompt v1. You will then inspect the model's answer.

In this step, get ready to:
  • Build a three-node manual workflow.
  • Configure one customer message for prompt v1.
  • Verify the model's answer in the execution view.
Build the manual input path

A Manual Trigger gives you control over when the workflow runs. An Edit Fields node supplies a consistent customer message for that run.

  • Select the workflow name at the top of the editor.
  • Replace the current name with Prompt Regression Harness.
  • Confirm that Prompt Regression Harness is visible at the top of the editor.

Your workflow now has a name that separates this prompt test from every other automation in your workspace.

  • Select the add-node control on the empty canvas.
  • Search for Manual Trigger.
  • Select Manual Trigger from the results.

You should see a Manual Trigger node on the canvas. This node becomes the starting point for every test run.

  • Select the add-node control after the Manual Trigger node.
  • Search for Edit Fields.
  • Select Edit Fields from the results.

The canvas should now show Manual Trigger connected to Edit Fields.

  • Keep Mode set to Manual Mapping.
  • Add a field inside the Fields to Set area.
  • Enter message as the field name.
  • Enter I need to reset my password as the field value.
  • Select Execute step in the Edit Fields node.

You should see one item in the output. Its message field contains I need to reset my password.

Why use these two nodes?

  • Manual Trigger starts the workflow only when you choose to run it.
  • Edit Fields creates a controlled input that you can reuse.
  • A fixed input gives future prompt versions the same starting point.

Message missing from the output?

  • Confirm that Mode is set to Manual Mapping.
  • Check that the field name is exactly message.
  • Run the Edit Fields node again after correcting the field.

Still stuck? Help me configure an n8n Edit Fields node that outputs one message field.

Configure prompt v1

The OpenAI Chat Model receives the customer message from the previous node. Prompt v1 gives the model one strict classification rule.

  • Return to the workflow canvas.
  • Select the add-node control after the Edit Fields node.
  • Search for OpenAI Chat Model.
  • Select OpenAI Chat Model from the results.

Your canvas should now show three connected nodes. Manual Trigger comes first. Edit Fields comes next. OpenAI Chat Model completes the path.

  • Select the existing OpenAI credential in the credential control.
  • Map the incoming message field into the user-message input.
  • Paste prompt v1 into the prompt field by using this exact text:
Return APPROVE only when the customer message contains the exact word reset and refers to a password. Otherwise return REJECT. Reply with only APPROVE or REJECT.

What does this prompt do?

  • The output APPROVE requires the exact word reset.
  • The message must also refer to a password.
  • Every other message receives REJECT.
  • The final sentence limits the answer to one of two values.
  • Check that the credential control shows OpenAI.
  • Confirm that the user-message input contains the mapped message field.
  • Return to the workflow canvas.

The completed path should show Manual Trigger connected to Edit Fields. Edit Fields should connect to OpenAI Chat Model.

Credential or prompt missing?

  • Confirm that you selected the existing OpenAI credential from earlier.
  • Check that the prompt field contains all three sentences.
  • Confirm that the mapped field is named exactly message.

Need another pair of eyes? Help me check the OpenAI Chat Model configuration in my n8n workflow.

Run the workflow and read the answer

A manual execution sends the same controlled message through the full path. The execution result shows whether the model followed your rule.

Before you run the workflow, what answer do you expect for this message? Hold that prediction while you start the run.

  • Select Execute Workflow in the workflow editor.

You should see each node complete successfully. The workflow passes one item from Manual Trigger to Edit Fields before sending it to OpenAI Chat Model.

  • Select the OpenAI Chat Model node in the completed execution.
  • Select OUTPUT in the node panel.

You should see APPROVE in the model output. The message contains the exact word reset and refers to a password.

That first green run gives you a working baseline. You can now see the input that entered the model and the answer that came back.

No AI answer in the output?

  • Confirm that the OpenAI Chat Model node shows the existing OpenAI credential.
  • Check that Edit Fields still outputs one non-empty message value.
  • Run the workflow again after correcting the missing configuration.

Still no result? Help me diagnose an n8n OpenAI workflow that runs without returning an answer.

✔️ Awesome, I've got everything!

Your active draft workflow is named Prompt Regression Harness. Its execution results show one AI answer for the customer message.

ⓧ I'd like to double check my workflow

  • Confirm that the workflow is named Prompt Regression Harness.
  • Confirm that Manual Trigger connects to Edit Fields.
  • Confirm that Edit Fields supplies I need to reset my password in a field named message.
  • Confirm that OpenAI Chat Model uses the existing OpenAI credential.
  • Confirm that the execution output contains an AI answer.
  • Compare the prompt field with this exact text:
Return APPROVE only when the customer message contains the exact word reset and refers to a password. Otherwise return REJECT. Reply with only APPROVE or REJECT.

What this confirms

This prompt matches the complete v1 rule for the workflow. The model receives one customer message and returns one classification.

Your one-message prompt path works from trigger to answer. Next, you will turn that single input into a fixed list of test cases.

Add Your Test Cases

Your first run proved that one customer message can travel through the OpenAI model. It also produced a decision you could inspect.

A single result cannot reveal how the prompt behaves across different wording. This step gives the Prompt Regression Harness a fixed set of eight cases.

n8n treats each case as an item. The same prompt can then process every case during one execution.

In this step, get ready to:
  • Replace the single message with eight fixed test cases.
  • Split the test-case array into individual n8n items.
  • Send all eight items through prompt v1 during one execution.
Build the fixed test list

A test suite needs inputs plus expected outcomes. Storing both in a JSON array creates a baseline that stays fixed while the prompt changes.

  • Return to the Editor view for Prompt Regression Harness.
  • Select the Edit Fields node that supplied one message earlier.
  • Set Mode to JSON Output.
  • Replace the contents of JSON Output with this configuration:
{
  "testCases": [
    {"testCaseId": "TC01", "message": "I need to reset my password", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC02", "message": "Please reset my password", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC03", "message": "I forgot my password and cannot sign in", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC04", "message": "My account is locked and I need access again", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC05", "message": "How do I change my email address?", "expectedOutcome": "REJECT"},
    {"testCaseId": "TC06", "message": "Reset my marketing preferences", "expectedOutcome": "REJECT"},
    {"testCaseId": "TC07", "message": "My password reset link expired", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC08", "message": "Why was I charged twice?", "expectedOutcome": "REJECT"}
  ]
}

How is each case structured?

  • The testCaseId field gives each case a stable identifier.
  • The message field contains the customer wording sent to the model.
  • The expectedOutcome field records the decision the prompt should return.
  • Select Execute step in the node panel.

You should see one output item containing a testCases array with eight entries. The array starts with TC01 and ends with TC08.

Test list not accepted?

  • Check that a comma separates every test-case object.
  • Confirm that the outer object starts with an opening brace.
  • Confirm that the outer object ends with a closing brace.

Still blocked? Help me fix my n8n test-case JSON.

Split the array into eight items

The array currently sits inside one n8n item. The Split Out node expands that list so each test case becomes independently processable.

  • Return to the workflow canvas.
  • Select the connection between the first Edit Fields node and the OpenAI Chat Model node.
  • Use the plus control on the connection to insert a node.
  • Search for Split Out.
  • Select Split Out from the results.
  • Enter testCases in Field to Split Out.
  • Select No Other Fields under Include.
  • Select Execute step in the Split Out panel.

You should see eight output items. The first carries TC01 inside testCases.

You have turned one bundled dataset into eight independently processable cases.

Only seeing one item?

  • Confirm that Field to Split Out contains testCases with the same capital letters.
  • Return to the first Edit Fields node to confirm its output contains eight array entries.

Need another pair of eyes? Help me find why Split Out returns one item.

Expose the fields and run all cases

Split Out keeps each case inside the testCases field. A second Edit Fields node lifts the three values to the top level for the model run.

  • Select the connection between Split Out and OpenAI Chat Model.
  • Use the plus control on the connection to insert a node.
  • Search for Edit Fields.
  • Select Edit Fields from the results.
  • Set Mode to Manual Mapping.
  • Create the string field testCaseId with the value {{ $json.testCases.testCaseId }}.
  • Create the string field message with the value {{ $json.testCases.message }}.
  • Create the string field expectedOutcome with the value {{ $json.testCases.expectedOutcome }}.

Why flatten the fields?

Top-level fields give the OpenAI Chat Model direct access to the current message. They also keep the case identifier beside the expected outcome.

That shared item becomes the audit trail for each model result.

  • Enable Keep Only Set Fields.
  • Select Execute step in the second Edit Fields panel.

You should see eight output items. Each item should contain testCaseId, message, and expectedOutcome as top-level fields.

Seeing empty mapped fields?

  • Check that each expression starts with $json.testCases.
  • Confirm that the Split Out output still contains a nested testCases object.
  • Check the capital letters in testCaseId and expectedOutcome.

Still seeing blank values? Help me debug my Edit Fields mappings.

The OpenAI credential and prompt v1 stay unchanged. The customer message remains mapped from {{ $json.message }}.

This execution sends exactly eight short model requests. That fixed limit keeps your OpenAI usage bounded.

Before you run the workflow, how many AI results do you expect one execution to produce?

  • Run all eight cases by selecting Execute Workflow.
  • Select the OpenAI Chat Model node after the execution completes.
  • Inspect the items in the OUTPUT panel.

You should find eight AI results covering TC01 through TC08. Each result retains its message plus its expected outcome.

That is the harness running at test-suite scale. One execution now exercises the same prompt eight times.

Fewer than eight AI results?

  • Select Split Out to confirm it produced eight items during the same execution.
  • Select the second Edit Fields node to confirm it passed eight items forward.
  • Confirm that the OpenAI Chat Model remains connected after the second Edit Fields node.

Need help tracing the missing item? Help me debug my eight-case n8n execution.

✔️ Awesome, I've got everything!

Your workflow now holds eight fixed cases. One execution sends every case through prompt v1.

ⓧ I'd like to double check the full code

Compare your workflow with this complete configuration for the current step.

{
  "testCases": [
    {"testCaseId": "TC01", "message": "I need to reset my password", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC02", "message": "Please reset my password", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC03", "message": "I forgot my password and cannot sign in", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC04", "message": "My account is locked and I need access again", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC05", "message": "How do I change my email address?", "expectedOutcome": "REJECT"},
    {"testCaseId": "TC06", "message": "Reset my marketing preferences", "expectedOutcome": "REJECT"},
    {"testCaseId": "TC07", "message": "My password reset link expired", "expectedOutcome": "APPROVE"},
    {"testCaseId": "TC08", "message": "Why was I charged twice?", "expectedOutcome": "REJECT"}
  ]
}

What should this contain?

This is the complete fixed test list for the first Edit Fields node. It contains the eight target cases without changing their wording or expected outcomes.

  • The canvas order is Manual Trigger, first Edit Fields, Split Out, second Edit Fields, and OpenAI Chat Model.
  • Split Out uses testCases as its Field to Split Out value.
  • The second Edit Fields node exposes testCaseId, message, and expectedOutcome.
  • The OpenAI Chat Model uses the existing OpenAI credential.
  • Prompt v1 says: 'Return APPROVE only when the customer message contains the exact word reset and refers to a password. Otherwise return REJECT. Reply with only APPROVE or REJECT.'
  • The completed execution contains eight AI results.

Your fixed test suite now runs every case through the same prompt. Next up, you will add a checker that turns those model answers into visible pass or fail results.

Check Every Result

Your OpenAI Chat Model now produces eight responses from the same fixed test list. Raw responses still leave you comparing every answer by eye.

A checking node in n8n turns each response into a repeatable verdict. It normalizes the model output before comparing it with expectedOutcome.

Each item then carries an actualOutcome field. It also carries a status field that shows PASS or FAIL.

In this step, get ready to:
  • Add a Code node after the model.
  • Normalize every response to APPROVE or REJECT.
  • Inspect the cases that prompt v1 gets wrong.
Add the result checker

The Code node receives all eight items in one execution. Its job is to preserve each test case while adding a consistent verdict.

  • Return to the workflow canvas from the completed execution.
  • Select the output connector on OpenAI Chat Model.
  • Search for Code in the node picker.
  • Select the Code node.
  • Rename the new node to Check Result.
  • Confirm the Mode field shows Run Once for All Items.
  • Replace the starter code in the editor by pasting this code:
// Check every input item from the model in one run.
return $input.all().map((item) => {
  const source = item.json;

  // Exclude fixed test fields so only the model result is checked.
  const modelResult = Object.fromEntries(
    Object.entries(source).filter(
      ([key]) => !['testCaseId', 'message', 'expectedOutcome'].includes(key),
    ),
  );

  // Reduce the model result to one expected label.
  const normalizedResponse = JSON.stringify(modelResult).toUpperCase();
  const matchedOutcome = normalizedResponse.match(/\b(APPROVE|REJECT)\b/);
  const actualOutcome = matchedOutcome ? matchedOutcome[1] : 'REJECT';

  // Preserve the test case while adding its verdict.
  item.json.actualOutcome = actualOutcome;
  item.json.status =
    actualOutcome === source.expectedOutcome ? 'PASS' : 'FAIL';

  return item;
});

What does this code do?

  • The $input.all() method collects every item received by the Code node.
  • The modelResult object excludes the fixed test fields. This prevents expectedOutcome from influencing the detected model result.
  • The normalizedResponse value converts the remaining model data to uppercase text.
  • The outcome match extracts either APPROVE or REJECT.
  • The status comparison adds PASS when both outcomes match. A mismatch receives FAIL.
  • Return to the workflow canvas after n8n accepts the code.
  • Confirm Code: Check Result remains directly connected after OpenAI Chat Model.

This rerun sends the same eight short cases through your configured model. The fixed list keeps the batch capped at eight.

Before you run the workflow, do you expect every model result to match its fixed expected outcome?

  • Start a fresh workflow execution from Manual Trigger.
  • Select Code: Check Result after the execution finishes.

You should see eight output items. Each item now has an actualOutcome field plus a status field.

The run produces six PASS rows. It also produces two FAIL rows.

You now have a measurable baseline for prompt v1.

Do you see fewer than eight checked items?

  • Confirm Code: Check Result connects directly after OpenAI Chat Model.
  • Confirm the Code node uses Run Once for All Items.
  • Compare your Code node with the complete reference below if actualOutcome or status is missing.

Still stuck? Help me debug why my n8n Code node is not returning eight items with actualOutcome and status fields.

✔️ Awesome, I've got everything!

Your Code: Check Result node now checks all eight model outputs. Double check that the latest execution contains six passes plus two failures.

ⓧ I'd like to double check the full code

Compare the complete code in Code: Check Result with this reference.

// Check every input item from the model in one run.
return $input.all().map((item) => {
  const source = item.json;

  // Exclude fixed test fields so only the model result is checked.
  const modelResult = Object.fromEntries(
    Object.entries(source).filter(
      ([key]) => !['testCaseId', 'message', 'expectedOutcome'].includes(key),
    ),
  );

  // Reduce the model result to one expected label.
  const normalizedResponse = JSON.stringify(modelResult).toUpperCase();
  const matchedOutcome = normalizedResponse.match(/\b(APPROVE|REJECT)\b/);
  const actualOutcome = matchedOutcome ? matchedOutcome[1] : 'REJECT';

  // Preserve the test case while adding its verdict.
  item.json.actualOutcome = actualOutcome;
  item.json.status =
    actualOutcome === source.expectedOutcome ? 'PASS' : 'FAIL';

  return item;
});
Inspect the failing cases

A failed check points to a specific gap between the intended behavior and the prompt wording. Each row preserves the original customer message for inspection.

Before you filter the table, which requests do you think the literal reset rule rejects?

  • Set the output table filter so status equals FAIL.
  • Open the row for TC03.
  • Compare its actualOutcome with its expectedOutcome.

For TC03, you should see an expected outcome of APPROVE. Its actual outcome is REJECT.

  • Open the row for TC04.
  • Compare its actualOutcome with its expectedOutcome.

For TC04, you should also see an expected outcome of APPROVE. Its actual outcome is REJECT.

Why does prompt v1 fail here?

Prompt v1 uses the exact word reset as a gate. TC03 describes a forgotten password without that word.

TC04 describes a locked account without that word. It also leaves the password context implied.

Both cases have an expectedOutcome of APPROVE. Their REJECT results expose the literal-word gap.

  • Clear the status filter.
  • Count the PASS values in the output table.
  • Count the FAIL values in the output table.

You should finish with six PASS rows plus two FAIL rows. TC03 and TC04 are the only failures.

Your v1 baseline now names the exact cases that broke. Next up, you will revise the prompt before rerunning the same eight cases to measure the change.

Improve the Prompt

Your n8n checker has exposed the weak spot in prompt v1. TC03 and TC04 describe password-access intent without the exact word reset.

The OpenAI model needs a broader decision rule. You will update only the prompt. The fixed test list makes the new pass count a fair comparison.

In this step, get ready to:
  • Identify why TC03 and TC04 failed.
  • Replace prompt v1 with an intent-based v2 prompt.
  • Run prompt v2 against the same eight cases.
Turn the failures into prompt rules

Each failed case points to one missing rule. TC03 captures forgotten-password language.

TC04 captures locked-account language. Both messages belong to password-reset support.

  • Review TC03 in the execution data already on screen.
  • Compare its actualOutcome with its expectedOutcome.
  • Review TC04 in the same execution data.
  • Compare its actualOutcome with its expectedOutcome.
  • Keep the testCases array unchanged for the next run.

Both rows show REJECT for actualOutcome even though expectedOutcome is APPROVE. That mismatch defines the two rules prompt v2 must add.

Why keep the test list fixed?

The same inputs act as a control for the comparison. A changed pass count can now be traced to the prompt edit.

Replace prompt v1

Prompt v1 depends on exact wording. Prompt v2 names the intent patterns that count as password-reset help.

  • Return to the workflow canvas for Prompt Regression Harness.
  • Select the OpenAI Chat Model node.
  • Find the prompt text that currently reads:
Return APPROVE only when the customer message contains the exact word reset and refers to a password. Otherwise return REJECT. Reply with only APPROVE or REJECT.

Why the current prompt misses the cases

The current rule approves a request only when it contains the exact word reset and refers to a password. That condition excludes TC03 and TC04.

  • Replace the entire prompt with this exact v2 text:
Classify customer messages. Return APPROVE when the customer needs password-reset help, including asking to reset a password, forgetting a password, being unable to sign in because of a password, or being locked out. Return REJECT for all other requests, including marketing-preference resets. Reply with only APPROVE or REJECT.

How the new prompt changes the rule

  • APPROVE covers requests to reset a password.
  • APPROVE covers forgotten-password requests.
  • APPROVE covers password-related sign-in failures.
  • APPROVE covers locked-out requests.
  • REJECT still covers unrelated requests such as marketing-preference resets.
  • The final instruction limits the response to one outcome label.
  • Confirm the prompt field contains the complete v2 text.
Re-run the fixed test suite

A fair regression check changes one variable. Here, only the prompt version changes.

Before you run this check, which failing cases do you expect the broader intent rules to fix?

  • Run the workflow from its Manual Trigger node.
  • Select the Code: Check Result node after the execution completes.
  • Inspect status across all eight output items.

You should see eight PASS values. No item has FAIL.

TC03 and TC04 now produce APPROVE. Every other case still matches its expected outcome.

Prompt v1 finished at six PASS results and two FAIL results. Prompt v2 finishes at eight PASS results and zero FAIL results.

You have closed both known gaps without breaking the six cases that already passed.

Still seeing a FAIL row?

  • Compare the prompt in OpenAI Chat Model with the complete v2 text above.
  • Confirm Edit Fields still contains all eight original test cases.
  • Run the workflow once more if the exact prompt produced an unexpected label.

Still stuck? Help me diagnose why my n8n prompt regression workflow still has a failing case.

Your fixed suite now proves that prompt v2 covers every known case. Next up, you will record each run so the prompt versions can be compared side by side.

Record Your Prompt Runs

Your v2 prompt now clears all eight tests. That result becomes more useful when you can compare it with the six-pass baseline from v1.

A run log turns each execution into evidence. In this step, you will send every checked item from n8n to Google Sheets so the two prompt versions stay comparable.

In this step, get ready to:
  • Create a run table with one column for each result field.
  • Connect the checked workflow output to the run table.
  • Record the eight cases from both prompt versions.
Create the run table

Each row needs enough context to reconstruct one test result. A stable set of columns keeps both prompt runs comparable.

  • Open the Google Sheets home screen in your browser.
  • Click New to create a spreadsheet.
  • Click the spreadsheet title at the top.
  • Enter Prompt Regression Runs as the spreadsheet title.
  • Double-click the worksheet tab at the bottom.
  • Enter Runs as the worksheet name.
  • Fill row 1 from A1 through F1 with these headers in order: promptVersion, testCaseId, message, expectedOutcome, actualOutcome, status.

You should see six named columns across the first row. This header row gives n8n a destination for every result field.

What Does Each Row Prove?

  • The promptVersion value identifies the classifier instructions used for the run.
  • The testCaseId value identifies the fixed test case.
  • The message value preserves the exact customer input.
  • The expectedOutcome value records the intended classification.
  • The actualOutcome value records the model response.
  • The status value makes each regression visible as PASS or FAIL.
Append checked rows with n8n

The checker already outputs one item per test case. An append operation can turn each incoming item into one new worksheet row.

  • Return to the workflow canvas for Prompt Regression Harness.
  • Select the connector after Code: Check Result.
  • Search for Google Sheets.
  • Select Google Sheets to add the node.

Connecting your Google account can feel sensitive. Your credential path depends on whether n8n provides managed OAuth in your workspace.

✔️ I see Sign in with Google

  • Choose the option to create a Google Sheets credential under Credential to connect with.
  • Select Sign in with Google.
  • Choose the Google account that can edit Prompt Regression Runs.
  • Approve the requested access after reviewing it.

You should return to the node with the new Google Sheets credential selected.

ⓧ I use self-hosted n8n

Self-hosted n8n requires a custom OAuth2 credential. The setup connects your n8n instance to a project in Google Cloud.

Google can now authorize spreadsheet access for your n8n instance.

  • Configure the OAuth consent screen for the project.
  • Create an OAuth client ID with Web application as the application type.
  • Copy the OAuth Redirect URL from the n8n credential.
  • Paste the URL into Authorized redirect URIs in Google Cloud.
  • Enter the generated Client ID in the n8n credential.
  • Enter the generated Client Secret in the n8n credential.
  • Select Sign in with Google.
  • Save the connected credential.

The selected credential should now show as connected. That confirms n8n can reach the spreadsheet with your permission.

  • Set Resource to Sheet Within Document.
  • Set Operation to Append Row.

You should now see fields for choosing a document and worksheet.

  • Choose From list in Document.
  • Select Prompt Regression Runs as the document.
  • Choose From list in Sheet.
  • Select Runs as the sheet.

n8n should load the six worksheet headers as available mapping fields.

Why Use Append Row?

Append Row creates a new row at the end of the worksheet for each incoming item. Your eight checked items therefore become eight separate records without overwriting an earlier run.

The column mapping is the fiddly part because every field name must line up with its worksheet header.

  • Set Mapping Column Mode to Map Each Column Manually.
  • Enter v1 for the promptVersion column.
  • Map the incoming testCaseId field to the matching worksheet column.
  • Map the incoming message field to the matching worksheet column.
  • Map the incoming expectedOutcome field to the matching worksheet column.
  • Map the incoming actualOutcome field to the matching worksheet column.
  • Map the incoming status field to the matching worksheet column.
  • Rename the node to Google Sheets: Append Run Row.

The node is now ready to turn the checker output into spreadsheet rows.

  • Save the workflow.

Cannot See the Spreadsheet?

  • Re-select From list after confirming the Google Sheets credential is connected.
  • Confirm that the connected Google account can edit Prompt Regression Runs.
  • Confirm that the worksheet is named Runs.

Still stuck? Help me connect my n8n Google Sheets node to the correct spreadsheet and worksheet.

Backfill and compare both runs

Both prompt runs happened before the append node existed. Data pinning lets you reuse the saved checker output without sending the eight messages through the model again.

  • Select the Executions tab in the workflow.
  • Open the earlier successful execution that contains six PASS items and two FAIL items.
  • Open Code: Check Result in that execution.
  • Select JSON in the OUTPUT view.
  • Copy the full output array with Copy Value.
  • Return to the workflow editor.
  • Open Code: Check Result.
  • Select JSON in the OUTPUT view.

The copied array holds all eight baseline results. Pinning it makes that exact dataset available to the new append node.

  • Select Edit.
  • Replace the existing output with the copied v1 array.
  • Select Save to pin the output.

You should see the This data is pinned banner above the checker output.

Before you append the baseline, how many rows do you expect the worksheet to gain?

  • Open Google Sheets: Append Run Row.
  • Select Execute step.

n8n should report eight output items from the append step.

  • Switch to the Runs worksheet in Google Sheets.

You should see eight v1 rows. Six rows show PASS, while TC03 and TC04 show FAIL.

No Baseline Rows in the Sheet?

  • Confirm that Code: Check Result shows eight pinned items.
  • Confirm that the append node targets Prompt Regression Runs.
  • Confirm that the append node targets the Runs worksheet.

Need another pair of eyes? Help me find why my pinned n8n items were not appended to Google Sheets.

Your baseline is stored. The same process now records the all-pass run.

  • Return to the n8n workflow.
  • Select the Executions tab.
  • Open the successful execution that contains eight PASS items.
  • Open Code: Check Result in that execution.
  • Select JSON in the OUTPUT view.
  • Copy the full output array with Copy Value.
  • Return to the workflow editor.
  • Open Code: Check Result.

The second copied array contains the same eight case IDs with the corrected outcomes.

  • Select JSON in the OUTPUT view.
  • Select Edit.
  • Replace the pinned output with the copied v2 array.
  • Select Save.
  • Open Google Sheets: Append Run Row.
  • Change the promptVersion value from v1 to v2.

Before you append this run, what should the total row count become after eight more results arrive?

  • Select Execute step.

n8n should report another eight output items from the append step.

  • Switch to the Runs worksheet in Google Sheets.

You should now see sixteen data rows. The eight v1 rows contain six PASS results and two FAIL results. The eight v2 rows all show PASS.

  • Return to the n8n workflow editor.
  • Open Code: Check Result.
  • Select Unpin in the pinned-data banner.
  • Save the workflow.
  • Return to the Runs worksheet.
  • Compare the two TC03 rows.
  • Compare the two TC04 rows.

Both cases move from FAIL under v1 to PASS under v2. That is your regression evidence captured in one table.

✔️ Awesome, I've got everything!

Your workflow now appends checked results to the Runs worksheet. The table contains both prompt versions for comparison.

ⓧ I'd like to double check the workflow

Use this checklist to compare your finished workflow with the target state.

  • Confirm that Manual Trigger connects to Edit Fields (testCases array).
  • Confirm that Edit Fields (testCases array) connects to Split Out (testCases).
  • Confirm that Split Out (testCases) connects to Edit Fields (testCaseId, message, expectedOutcome).
  • Confirm that Edit Fields (testCaseId, message, expectedOutcome) connects to OpenAI Chat Model.
  • Confirm that OpenAI Chat Model connects to Code: Check Result.
  • Confirm that Code: Check Result connects to Google Sheets: Append Run Row.
  • Confirm that the append node writes promptVersion, testCaseId, message, expectedOutcome, actualOutcome, and status to Runs.
  • Confirm that the worksheet contains eight rows for each prompt version.

Secret mission

Make a Prompt Release Decision

A perfect score is useful when you can explain why it earns release approval. Compare v1 with v2 to make the ship decision. Draft the next edge case without changing the original baseline.

Clean Up Your Resources

Clean Up Your Resources

Your n8n workflow creates OpenAI API usage only when you run it. Your Google Sheets file stays idle between runs.

Choose whether to keep these resources available, pause your testing, or delete the project entirely.

Cost warning

Each future workflow run sends all eight test cases to the model. Those model calls can add API usage charges.

  • Choose Pause or Delete if you do not plan to test another prompt version soon.

Resources you used:

  • An active n8n workflow named Prompt Regression Harness with eight fixed test cases.
  • A Google Sheet named Prompt Regression Runs with the Runs worksheet.
  • An OpenAI credential stored in n8n for model calls.
  • The OpenAI API key represented by that credential.
  • A Google Sheets credential stored in n8n for appending run rows.

Keep everything running

No action is needed. Choose this option if you want to keep testing future prompt versions against the same cases.

  • Keep Prompt Regression Harness in n8n.
  • Keep the eight fixed test cases unchanged so future pass counts remain comparable.
  • Keep Prompt Regression Runs in Google Sheets.
  • Add a fixed test case whenever you discover another prompt failure.
  • Keep the OpenAI credential in n8n only if you plan to reuse it.
  • Keep the OpenAI API key only if you plan to run more model tests.
  • Keep the Google Sheets credential in n8n only if you plan to append more run rows.

Pause - I'll come back to this later

Pause new test activity while preserving the workflow. Your comparison rows remain available for your next prompt edit.

  • Switch back to Prompt Regression Harness in n8n.
  • Turn off the workflow's active state.
  • Confirm the workflow now shows an inactive state.
  • Leave the eight fixed test cases unchanged.
  • Leave Prompt Regression Runs in Google Sheets.
  • Leave the OpenAI credential in n8n if you plan to resume testing.
  • Leave the Google Sheets credential in n8n if you plan to append more results.

Delete - I don't want to use this again

Deleting project resources is permanent. Shared credentials may support other workflows.

  • Return to Prompt Regression Harness in n8n.
  • Delete the workflow with its workflow deletion control.
  • Confirm Prompt Regression Harness no longer appears in your n8n workflow list.
  • Return to Prompt Regression Runs in Google Sheets.
  • Move the spreadsheet to trash with its file deletion option.
  • Confirm Prompt Regression Runs no longer appears in your Google Sheets file list.

Credential cleanup needs extra care. Delete only the credentials created for this project.

  • Return to the OpenAI credential from earlier in n8n.
  • Delete the OpenAI credential from n8n if you created it only for this project.
  • Revoke the matching API key in your OpenAI account if you created it only for this project.
  • Return to the Google Sheets credential from earlier in n8n.
  • Delete the Google Sheets credential from n8n if you created it only for this project.
  • Confirm the project-only credentials no longer appear in n8n.

Nice Work!

Nice Work!

You did it! Your n8n workflow now catches prompt regressions across eight fixed cases before the prompt ships.

You've learned how to:

  • Build a prompt regression harness in n8n that evaluates eight fixed customer-message cases.
  • Refine the prompt from a six-pass v1 result to an eight-pass v2 result by following the failed cases.
  • Store a Google Sheets run history that compares every expectedOutcome with its actualOutcome across both prompt versions.
  • Secret Mission: Extend the prompt testing workflow through an optional challenge that pushes your skills further.

Ready to quiz yourself?