AI Test-Guard for Pull Requests

Build an AI reviewer that flags untested pull-request behavior.

Introduction

30 Second Summary

A green check can make a code change feel safe. It only confirms the checks you already wrote, so new behavior can still slip through uncovered.

In this project, you will build AI Test-Guard, a deployment-risk classifier protected by deterministic Bash tests. You will add a Gemini-powered pull-request reviewer that spots new behavior without matching test coverage.

What You'll Build

Your finished pull request in GitHub will show green deterministic CI beside an AI review confirming every changed behavior has a matching test.

By the end of this project, you'll have:

A deployment-risk classifier you can run against a changed path to see low, medium, high, or unknown.
A deterministic CI workflow that gives each push or pull request a visible green test result in GitHub Actions.
An AI-reviewed pull request that exposes a missing assertion while CI passes. The next review recognizes the test you add.
Secret Mission: Turn the advisory reviewer into a bounded review gate that requests changes when new behavior has no matching assertion.

Are there any prerequisites?

Basic Git knowledge plus a GitHub account, a code editor, a Google account, and familiarity with DevOps are the only prerequisites.

The setup step covers GitHub CLI, GitHub Agentic Workflows, and Google AI Studio before you automate the repository.

Before We Start

Before any setup, lock in the boundary for AI Test-Guard: it reviews pull-request behavioral test adequacy where deterministic CI cannot judge coverage. A human still decides whether to merge.

Prepare the Free Toolchain

AI Test-Guard needs authenticated local tools before it can connect to GitHub.

The toolchain must also be reproducible. This step pins its CLI versions while keeping the AI credential away from code.

In this step, get ready to:

Prepare Homebrew for GitHub CLI 2.99.0.
Authenticate GitHub CLI before pinning gh-aw to v0.88.4.
Store a new Google AI Studio Free Tier auth API key outside code.

Install and verify GitHub CLI

Homebrew manages the command-line tools used in this project. Checking it first gives GitHub CLI a consistent installation path.

Press Cmd+Space to open Spotlight Search.
Type Terminal into Spotlight Search.
Press Enter to open macOS Terminal.
Check whether Homebrew is available by running this command:

brew --version

What does this check do?

The version check asks brew to report the installed Homebrew release. A version confirms that Terminal can find Homebrew.

✔️ I see a Homebrew version

Homebrew is ready to manage GitHub CLI.

ⓧ Command not found

The installer can take several minutes while it prepares Homebrew. Terminal may stay busy during parts of the installation.

Install Homebrew by running the official installer command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

What does this installer do?

The curl command downloads the official Homebrew installer.
The /bin/bash shell runs that installer on your Mac.

Follow the shell setup instruction printed at the end of the installer.
Confirm that Homebrew is now available by running this command:

brew --version

What should this confirm?

Homebrew now prints its version instead of reporting a missing command. That output confirms the shell setup instruction worked.

Still missing Homebrew?

Return to the final lines of the installer output. Run the shell setup instruction exactly as Homebrew printed it.

Help me troubleshoot why Homebrew is still unavailable in macOS Terminal.

This project pins GitHub CLI to 2.99.0. The version check separates an exact match from an older installation or a missing command.

Check the installed GitHub CLI version by running this command:

gh --version

What does this check do?

The gh command reports the GitHub CLI version available in Terminal. The project uses 2.99.0 for a reproducible setup.

✔️ I see the required version

GitHub CLI 2.99.0 is already available. That part of the toolchain is locked in.

ⓧ I see an older version

Upgrade GitHub CLI with Homebrew before checking its version again by running:

brew upgrade gh
gh --version

What do these commands do?

Homebrew upgrades the installed gh formula.
The second command reports the version that Terminal now uses.

The final output should include 2.99.0.

Still seeing the older version?

Close Terminal after the upgrade. Open a new Terminal window so your shell can resolve the updated installation.

Help me find why macOS Terminal still uses an older GitHub CLI version.

ⓧ Command not found

Install GitHub CLI with Homebrew before checking its version by running:

brew install gh
gh --version

What do these commands do?

Homebrew installs the gh command on your Mac.
The second command confirms the installed version.

The final output should include 2.99.0.

GitHub CLI still unavailable?

Check that the Homebrew installation completed without a network error. Open a new Terminal window after the installation succeeds.

Help me troubleshoot a GitHub CLI installation through Homebrew.

Authenticate GitHub CLI and pin gh-aw

Authentication lets GitHub CLI work with repositories and workflows through your account. The requested scopes cover those two responsibilities.

Start the GitHub CLI authentication flow by running the command below:
Choose GitHub.com when Terminal asks which account to use.
Choose HTTPS when Terminal asks for a Git protocol.
Choose browser authentication when Terminal offers an authentication method.
Approve the requested access in your browser.

gh auth login --scopes repo,workflow

Why are these scopes needed?

The repo scope lets GitHub CLI work with your repositories.
The workflow scope lets GitHub CLI manage workflow files.

Confirm the authenticated account and granted scopes by running this command:

gh auth status

What should this prove?

The status check confirms which GitHub account the CLI uses. It also reports the scopes granted during authentication.

You should see your GitHub account listed as authenticated. The granted scopes should include repo and workflow.

Authentication did not complete?

Restart the login flow if the browser window closed before approval. Check that Terminal remains open while you complete the browser step.

Help me troubleshoot GitHub CLI authentication on macOS.

GitHub Agentic Workflows uses the gh-aw extension to compile Markdown instructions into GitHub Actions workflows. Pinning v0.88.4 keeps later compiler output reproducible.

Check whether gh-aw is already installed by running this command:

gh aw version

What does this check do?

The version check reports the gh-aw extension available through GitHub CLI. It also reveals whether a different release needs replacing.

✔️ I see the required gh-aw version

gh-aw reports v0.88.4. Your compiler is pinned to the project version.

ⓧ I see another version

Replace the existing extension with the pinned release before checking it by running:

gh extension remove gh-aw
gh extension install github/gh-aw@v0.88.4
gh aw version

What do these commands do?

GitHub CLI removes the currently installed gh-aw extension.
GitHub CLI installs the pinned v0.88.4 release from its public repository.
The final check reports the active gh-aw version.

The final output should report v0.88.4.

Pinned extension failed to install?

Confirm that GitHub CLI authentication still succeeds. Check your internet connection before retrying the pinned installation.

Help me troubleshoot the pinned gh-aw extension installation.

ⓧ gh-aw is not installed

Install the pinned gh-aw release before checking its version by running:

gh extension install github/gh-aw@v0.88.4
gh aw version

What do these commands do?

GitHub CLI installs gh-aw from the official public repository.
The version suffix selects the pinned v0.88.4 release.
The final check reports the installed extension version.

The final output should report v0.88.4.

Extension command still unavailable?

Confirm that the extension installation completed without a network or authentication error. Open a new Terminal window after it succeeds.

Help me troubleshoot why the gh-aw command is unavailable.

Create the Gemini API key

The later AI workflow needs an auth API key from Google AI Studio. A Free Tier project keeps this project inside the free-only budget.

Credentials can feel risky. This path moves the key straight into your password manager without passing through code or shell history.

Open the official Gemini API key guide in your browser.
Follow the guide into Google AI Studio.
Sign in with your Google account.
Select a project with Free Tier access.
Click Create API key.
Copy the generated key directly into your password manager.
Close the key display after the password manager confirms it is saved.

Keep the later repository synthetic

Free Tier prompts and responses may be used to improve Google products. The later repository uses synthetic code so private data stays out of the workflow.

Your auth API key now has a safe temporary home outside the project. No code or shell history contains the credential.

Before you run the final checks, which three signals would prove the toolchain is ready?

Return to Terminal.
Verify the CLI version, authentication state, and compiler version by running:

gh --version
gh auth status
gh aw version

What do these checks prove?

The first check proves the pinned GitHub CLI is available.
The second check proves GitHub CLI is connected to your account.
The third check proves the pinned gh-aw compiler is available.

The first output should include 2.99.0. The second should confirm your authenticated GitHub account.

The third output should report gh-aw v0.88.4. That is the full local toolchain working together.

Your free toolchain is ready. Next up, you will build a small Bash classifier that gives CI real behavior to test.

Build the Risk Checker

Your GitHub access is ready. AI Test-Guard now needs a small behavior that deterministic CI can check.

A Bash classifier keeps that behavior visible without package dependencies. You will store it in a local Git repository before publishing the baseline.

In this step, get ready to:

Create the local repository structure on your Desktop.
Classify documentation paths and application source paths.
Test the known behavior before publishing the baseline.

Create the local classifier

A deployment-risk classifier maps a changed file path to a risk label. Its fallback returns unknown when no known path pattern matches.

Create the ai-test-guard repository on your Desktop by running these commands:

cd ~/Desktop
mkdir ai-test-guard
cd ai-test-guard
git init -b main
mkdir -p scripts tests
ls

What do these commands create?

The first three commands create ai-test-guard on your Desktop.
The git init -b main command initializes the repository on the main branch.
The final commands create the scripts directory plus the tests directory.

Repository setup failed?

Check that your Terminal prompt is inside ~/Desktop before creating the folder.

If Git rejects the branch option, confirm that your installed Git version supports named initial branches.

Help me diagnose why my local Git repository setup failed.

Each quoted here-document writes its content exactly as shown. This protects the Bash variables from expansion while Terminal creates the files.

Create README.md inside ai-test-guard by running this command:

cat <<'EOF' > README.md
# AI Test-Guard

A small CI/CD demonstration that combines deterministic Bash tests with an AI pull-request reviewer.

## Run locally

```bash
bash tests/deployment-risk.test.sh
bash scripts/deployment-risk.sh infrastructure/network.tf
```

The deterministic workflow proves known assertions still pass. AI Test-Guard reviews pull-request diffs for new behavior that lacks a matching assertion.

EOF

What does the README capture?

The README names the project and records its local commands. It also explains how deterministic tests fit beside AI review.

Create scripts/deployment-risk.sh inside the scripts directory by running this command:

cat <<'EOF' > scripts/deployment-risk.sh
#!/usr/bin/env bash
set -euo pipefail

classify_path() {
  case "$1" in
    docs/*)
      printf 'low\n'
      ;;
    src/*)
      printf 'medium\n'
      ;;
    *)
      printf 'unknown\n'
      ;;
  esac
}

if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
  if [[ $# -ne 1 ]]; then
    printf 'Usage: %s <changed-path>\n' "$0" >&2
    exit 64
  fi

  classify_path "$1"
fi

EOF

What does this classifier do?

The classify_path() function compares its input with known path patterns.
The docs/* branch returns low for documentation changes.
The src/* branch returns medium for application source changes.
The fallback returns unknown for every unmatched path.

Demonstrate both known classifications by running these commands:

bash scripts/deployment-risk.sh docs/runbook.md
bash scripts/deployment-risk.sh src/api.sh

What should I see?

The first command prints low. The second command prints medium.

You now have a visible classifier that responds differently to two changed paths.

Classifier output missing?

Confirm that scripts/deployment-risk.sh sits inside the scripts directory.

Check the closing delimiter if Terminal appears to wait for more input. The final EOF must sit alone at the start of its line.

Help me fix my deployment-risk classifier.

Test the known behavior

Direct demonstrations show individual outputs. Assertions turn those expectations into repeatable checks that stop as soon as a known behavior changes.

Create tests/deployment-risk.test.sh inside the tests directory by running this command:

cat <<'EOF' > tests/deployment-risk.test.sh
#!/usr/bin/env bash
set -euo pipefail

source "$(dirname "$0")/../scripts/deployment-risk.sh"

assert_equal() {
  local expected="$1"
  local actual="$2"
  local description="$3"

  if [[ "$actual" != "$expected" ]]; then
    printf 'FAIL: %s. Expected %s, got %s\n' "$description" "$expected" "$actual" >&2
    exit 1
  fi
}

assert_equal "low" "$(classify_path "docs/runbook.md")" "documentation risk"
assert_equal "medium" "$(classify_path "src/api.sh")" "application source risk"

printf '2 tests passed\n'

EOF

How does the test suite work?

The source line loads classify_path() without running the script entry point.
The assert_equal() helper compares each actual label with its expected label.
The two assertions cover the current documentation behavior and application source behavior.
The final message appears only after both assertions pass.

Before you run the suite, do you expect both path examples to satisfy their assertions?

Run the complete test suite with this command:

bash tests/deployment-risk.test.sh

What does this result prove?

You will see 2 tests passed. This proves both known path classifications match their current assertions.

That is your first working slice. The classifier now has repeatable evidence for every behavior it currently supports.

Tests not passing?

Check that the test file is named deployment-risk.test.sh inside the tests directory.

Compare the expected labels with the exact lowercase output from classify_path().

Help me debug the failing Bash assertions.

✔️ Awesome, I've got everything!

Your classifier and its two tests are ready. Keep Terminal inside the ai-test-guard directory.

ⓧ I'd like to double check the full code

# AI Test-Guard

A small CI/CD demonstration that combines deterministic Bash tests with an AI pull-request reviewer.

## Run locally

```bash
bash tests/deployment-risk.test.sh
bash scripts/deployment-risk.sh infrastructure/network.tf
```

The deterministic workflow proves known assertions still pass. AI Test-Guard reviews pull-request diffs for new behavior that lacks a matching assertion.

#!/usr/bin/env bash
set -euo pipefail

classify_path() {
  case "$1" in
    docs/*)
      printf 'low\n'
      ;;
    src/*)
      printf 'medium\n'
      ;;
    *)
      printf 'unknown\n'
      ;;
  esac
}

if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
  if [[ $# -ne 1 ]]; then
    printf 'Usage: %s <changed-path>\n' "$0" >&2
    exit 64
  fi

  classify_path "$1"
fi

#!/usr/bin/env bash
set -euo pipefail

source "$(dirname "$0")/../scripts/deployment-risk.sh"

assert_equal() {
  local expected="$1"
  local actual="$2"
  local description="$3"

  if [[ "$actual" != "$expected" ]]; then
    printf 'FAIL: %s. Expected %s, got %s\n' "$description" "$expected" "$actual" >&2
    exit 1
  fi
}

assert_equal "low" "$(classify_path "docs/runbook.md")" "documentation risk"
assert_equal "medium" "$(classify_path "src/api.sh")" "application source risk"

printf '2 tests passed\n'

Publish the baseline

A commit preserves this working baseline as one snapshot. Publishing it creates the public repository that later automation can run against.

Stage and commit the baseline by running these commands:

git add .
git commit -m "Build deployment risk checker"

What does this commit contain?

The commit records README.md with both Bash files. It gives the project a known passing point before CI is added.

Commit blocked?

If Git asks for your identity, follow the printed instructions to configure the requested author details.

If Git reports no files to commit, confirm that Terminal is inside the ai-test-guard directory.

Help me fix my baseline commit.

This command makes the repository public. Your API key stays outside the project folder, so it is not included in the push.

Create the public repository and push the main branch by running this command:

gh repo create ai-test-guard --public --source=. --remote=origin --push

What does publishing configure?

GitHub CLI creates the public ai-test-guard repository. It configures origin before pushing your committed main branch.

Before the final check, what repository details do you expect GitHub CLI to return?

Verify the published repository by running this command:

gh repo view --json name,visibility,url

What should I see?

The output includes the repository name and its public visibility. It also includes the GitHub URL for your published baseline.

Repository details unavailable?

Confirm that the repository creation command completed before running the verification.

Check that Terminal remains inside the local ai-test-guard repository.

Help me diagnose why GitHub CLI cannot find my repository.

Your classifier is now public with two passing tests. The repository has a clean baseline for deterministic automation.

Your risk checker is ready. Next up, you will run its Bash tests automatically on every push and pull request.

Add Deterministic CI

Your Bash classifier already proves two known behaviors locally. The repository now needs a repeatable check that runs whenever code reaches GitHub.

A GitHub Actions workflow provides that deterministic CI baseline on every push or pull request. The later AI reviewer can focus on whether new behavior has enough test coverage.

In this step, get ready to:

Add a workflow that runs the Bash test suite on pushes and pull requests.
Confirm the workflow uses the same command that passes locally.
Publish the workflow to produce a green CI run.

Add CI

A workflow file tells GitHub which events should start automation. It also defines the permissions and steps available to the hosted runner.

In your code editor's file explorer, create .github/workflows/ci.yml inside ai-test-guard.

You'll see ci.yml nested under .github/workflows in the file explorer.

Configure the CI job by pasting this workflow into .github/workflows/ci.yml:

name: CI

on:
  push:
  pull_request:

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v7.0.1
      - name: Run deployment-risk tests
        shell: bash
        run: bash tests/deployment-risk.test.sh

What Does This Workflow Do?

The push trigger starts CI after a commit reaches the repository.
The pull_request trigger starts CI when a pull request changes.
The contents: read permission applies least privilege to repository access.
The ubuntu-latest label selects a hosted Ubuntu runner.
The actions/checkout@v7.0.1 step makes the repository files available to the test job.
The final step runs the existing Bash test file.

Save .github/workflows/ci.yml.
Inspect the uses line to confirm it contains actions/checkout@v7.0.1.
Inspect the run line to confirm it contains bash tests/deployment-risk.test.sh.

Workflow File in the Wrong Place?

Check that ci.yml is inside .github/workflows within the ai-test-guard repository.
Compare your indentation with the complete file below.
Ask for help checking my workflow path and indentation.

✔️ Awesome, I've got everything!

Your saved workflow contains both triggers, read-only access, the hosted runner, the pinned checkout action, and the Bash test step.

ⓧ I'd like to double check the full code

name: CI

on:
  push:
  pull_request:

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v7.0.1
      - name: Run deployment-risk tests
        shell: bash
        run: bash tests/deployment-risk.test.sh

Run the local baseline

The hosted workflow uses the same test command that already works on your machine. One more local run confirms the baseline is healthy before you publish its automation.

Before you run the suite, what result do you expect from the two existing assertions?

Run the existing test suite locally by running this command:

bash tests/deployment-risk.test.sh

What Does This Command Prove?

This command invokes the same test file named in ci.yml. A successful result proves the known assertions are ready for the hosted runner.

You'll see the output end with 2 tests passed.

Tests Not Passing Locally?

Confirm your terminal is inside the local ai-test-guard repository.
Compare tests/deployment-risk.test.sh with the two-test version from the previous step.
Help me diagnose why the two-test Bash suite is not passing.

Publish CI

GitHub can run the workflow after the file reaches the remote repository. The push becomes the first real test of your hosted CI configuration.

Stage .github/workflows/ci.yml in your code editor's source control panel.

Your source control panel now lists only .github/workflows/ci.yml as staged.

Enter Add deterministic CI as the commit message.
Commit the staged workflow to main using the source control controls.

Your branch history now includes the deterministic CI workflow.

Push main to origin using the source control sync control.

Before you watch the run, do you expect GitHub Actions to reproduce the local result?

Watch the triggered workflow from Terminal by running this command:

gh run watch --compact --exit-status

What Does This Command Do?

The watcher follows the workflow run from your terminal.
The --compact flag keeps the status output concise.
The --exit-status flag returns a failing status when the workflow fails.

You'll see Run deployment-risk tests complete successfully. You have now locked the two known assertions into a green CI baseline.

Workflow Run Not Green?

Wait until the pushed commit appears in the public repository.
Repeat the watch command after GitHub starts the workflow run.
Compare the failed test step with the command in .github/workflows/ci.yml.
Help me diagnose why my deterministic CI workflow is not green.

Visit the public ai-test-guard repository in your browser.
Select the Actions tab.
Open the latest CI run.
Confirm Run deployment-risk tests has a green success indicator.

Your repository now has a reliable baseline for every known assertion. Next, you'll expose what that green signal cannot prove.

Expose the Coverage Blind Spot

Your main branch now has a green GitHub Actions run. That run proves the existing Bash assertions pass.

Deterministic CI checks only the assertions it runs. This step measures what happens when behavior grows faster than its tests.

In this step, get ready to:

Create a local branch for the infrastructure-risk behavior.
Add a high-risk classification without changing the test suite.
Run the unchanged test suite against the new behavior.

Isolate the experiment

A Git branch keeps this deliberate coverage gap separate from your green main branch. The change stays local until the next step.

Return to the macOS Terminal window from earlier.
Create feature/infrastructure-risk and make it active by running this command:

git switch -c feature/infrastructure-risk

What does this command do?

The -c flag creates feature/infrastructure-risk.
Git makes the new branch active for your next commit.

Your terminal confirms that feature/infrastructure-risk is now active.

Branch not created?

Return to the ai-test-guard repository if your terminal is in a different folder.
Use the existing local branch if Git reports that the branch name already exists.

Ask for help with the terminal output: Help me diagnose why my feature branch was not created..

Extend the classifier

The classifier's case statement maps path patterns to risk labels. The infrastructure branch belongs before the wildcard path so it can match first.

Open scripts/deployment-risk.sh from the ai-test-guard repository in your code editor.
Find this existing src/* branch:

    src/*)
      printf 'medium\n'
      ;;

Why use this anchor?

This block marks the exact insertion point before the wildcard branch.
Keeping specific patterns above *) lets the classifier reach them.

Insert the infrastructure case below the src/* block so that the section looks like this:

    src/*)
      printf 'medium\n'
      ;;
    infrastructure/*)
      printf 'high\n'
      ;;

What does this branch add?

The infrastructure/* pattern covers changed paths below the infrastructure directory.
The high result marks those changes as high risk.

Save scripts/deployment-risk.sh.
Leave tests/deployment-risk.test.sh unchanged.

Unsure about the case order?

Check that infrastructure/* sits below src/*.
Check that infrastructure/* sits above *).

Get help checking the structure: Help me verify the order of my Bash case branches..

✔️ Awesome, I've got everything!

Your saved classifier now includes the infrastructure branch between the source and wildcard branches.

ⓧ I'd like to double check the full code

#!/usr/bin/env bash
set -euo pipefail

classify_path() {
  case "$1" in
    docs/*)
      printf 'low\n'
      ;;
    src/*)
      printf 'medium\n'
      ;;
    infrastructure/*)
      printf 'high\n'
      ;;
    *)
      printf 'unknown\n'
      ;;
  esac
}

if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
  if [[ $# -ne 1 ]]; then
    printf 'Usage: %s <changed-path>\n' "$0" >&2
    exit 64
  fi

  classify_path "$1"
fi

Reveal the coverage blind spot

The test file still contains assertions for docs/* and src/* only. Its next result measures those known behaviors.

Before you run the suite, do you think it can detect a missing assertion for a branch it never executes?

Test the changed classifier with the unchanged suite by running:

bash tests/deployment-risk.test.sh

What did this prove?

The command prints 2 tests passed because both existing assertions still pass.
The new infrastructure/* branch never runs during this suite.
A green result here covers the known assertions only.

You have captured the blind spot. The suite is green while the new high path has no matching assertion.

Tests did not pass?

Compare scripts/deployment-risk.sh with the full-code tab if Bash reports a syntax problem.
Confirm that tests/deployment-risk.test.sh still contains the two original assertions if the test count differs.

Get help with the failing run: Help me debug why the unchanged two-test Bash suite is failing..

A local commit freezes this exact comparison point. It contains the new behavior while the test file remains untouched.

Record only the classifier change in a local commit by running these commands:

git add scripts/deployment-risk.sh
git commit -m "Add infrastructure risk classification"

What does this commit capture?

The first command stages only scripts/deployment-risk.sh.
The second command records the untested infrastructure behavior on feature/infrastructure-risk.
No push command runs, so the commit remains local.

You should see Git confirm a new commit with one changed file. Your public main branch keeps its green CI result.

Commit missing the change?

Confirm that scripts/deployment-risk.sh is saved in your editor.
Confirm that your terminal still uses feature/infrastructure-risk.

Ask for help without discarding work: Help me diagnose why Git did not commit my classifier change..

Your local branch now carries a committed coverage gap. Next, you'll add a bounded reviewer that can reason about the gap in a pull request.

Add the AI Test-Guard

Your feature branch now contains a new infrastructure-risk behavior. The existing deterministic suite still passes with only two assertions.

That green result cannot tell whether the new branch has matching coverage. In this step, GitHub Agentic Workflows uses Gemini to review the diff under bounded safe outputs.

In this step, get ready to:

Store the Gemini credential as a repository Actions secret.
Publish a bounded advisory workflow on the main branch.
Bring deterministic CI into agreement with the AI review after the coverage finding.

Configure the bounded reviewer

The reviewer needs an API key to call Gemini. A GitHub Actions secret keeps that credential outside your files.

Switch back to the existing main branch by running this command:

git switch main

What does this command do?

This returns your local repository to the branch with the green CI workflow. The agentic workflow belongs on this default branch.

Credentials deserve a careful pause. This prompt keeps the key out of project files or shell history.

Start the repository secret prompt with GitHub CLI by running this command:

gh secret set GEMINI_API_KEY

How is the key stored?

The command stores the value as the repository secret named GEMINI_API_KEY. The workflow can use the secret without placing its value in a commit.

Paste the Google AI Studio key from your password manager into the secure prompt.
Press Enter to save the secret.

The command completes without displaying the stored key.

Secret prompt did not complete?

Confirm that GitHub CLI is still authenticated to the account that owns the repository.
Confirm that the secret name is exactly GEMINI_API_KEY.
Ask for targeted help with the repository secret command.

The agentic workflow is authored as Markdown with YAML frontmatter. The frontmatter limits its trigger, permissions, engine version, and allowed review output.

Create .github/workflows/ai-test-guard.md inside the existing .github/workflows folder with your code editor.
Add the workflow frontmatter by pasting this first block:

---
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: read

engine:
  id: gemini
  version: "0.58.0"

safe-outputs:
  add-comment:
    max: 1
  create-pull-request-review-comment:
    max: 5
  submit-pull-request-review:
    allowed-events: [COMMENT]
---

What does the frontmatter control?

The pull-request trigger runs when a pull request is opened or synchronized with a new commit.
The permissions give the agent read-only access to repository contents and pull-request data.
The engine uses Gemini CLI version 0.58.0.
The safe outputs permit one summary comment, up to five inline comments, and a COMMENT review.

Save .github/workflows/ai-test-guard.md.
Confirm that the frontmatter ends with a second --- line.

Frontmatter looks misaligned?

Check that each nested value uses two more spaces than its parent.
Check that COMMENT remains inside the square brackets.
Ask for help checking the workflow frontmatter.

The Markdown body defines the review policy in plain language. It focuses the agent on one bounded question about changed behavior and matching assertions.

Add the review instructions directly below the closing frontmatter line by pasting this block:

# AI Test-Guard

Review only the pull request diff and the repository tests.

Check whether every new behavior branch in `scripts/deployment-risk.sh` has a matching assertion in `tests/deployment-risk.test.sh`.

If coverage is missing:

- Create an inline review comment on the uncovered branch.
- Name the missing input and expected output.
- Submit a `COMMENT` review.
- Add one concise summary comment titled `Test coverage gap`.

If coverage is complete:

- Submit a `COMMENT` review stating that the changed behavior has a matching test.
- Add one concise summary comment.

Do not modify code, push commits, approve, request changes, or merge the pull request.

How is the review bounded?

The prompt restricts the review to the pull-request diff and two named files. Its safe outputs provide validated review operations without direct write permission.

The reviewer can report a coverage gap. It cannot edit code, push commits, approve changes, request changes, or merge the pull request.

Save .github/workflows/ai-test-guard.md.
Confirm that the final line prevents the reviewer from modifying or merging code.

Workflow instructions look incomplete?

Confirm that both file paths use the exact names from the repository.
Confirm that both coverage outcomes include a summary comment.
Ask for help comparing the review policy.

Compare your completed workflow source with the reference below.

✔️ Awesome, I've got everything!

Your source file now defines the advisory reviewer. Double-check that you saved it before compiling.

ⓧ I'd like to double check the full code

---
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: read

engine:
  id: gemini
  version: "0.58.0"

safe-outputs:
  add-comment:
    max: 1
  create-pull-request-review-comment:
    max: 5
  submit-pull-request-review:
    allowed-events: [COMMENT]
---

# AI Test-Guard

Review only the pull request diff and the repository tests.

Check whether every new behavior branch in `scripts/deployment-risk.sh` has a matching assertion in `tests/deployment-risk.test.sh`.

If coverage is missing:

- Create an inline review comment on the uncovered branch.
- Name the missing input and expected output.
- Submit a `COMMENT` review.
- Add one concise summary comment titled `Test coverage gap`.

If coverage is complete:

- Submit a `COMMENT` review stating that the changed behavior has a matching test.
- Add one concise summary comment.

Do not modify code, push commits, approve, request changes, or merge the pull request.

What should match?

Compare the frontmatter limits and the complete review policy with your saved file. Preserve every permission and safe-output value exactly.

Before you compile, what generated workflow file do you expect to appear beside the Markdown source?

Compile the agentic workflow with the pinned gh-aw installation by running this command:

gh aw compile

What does compilation produce?

The compiler validates the Markdown source. It generates .github/workflows/ai-test-guard.lock.yml as the executable GitHub Actions workflow.

The generated file comes from gh-aw version v0.88.4. Keep it committed without manual edits.

Inspect .github/workflows/ai-test-guard.md in your code editor.
Inspect the generated .github/workflows/ai-test-guard.lock.yml file in the same folder.
Confirm that both workflow files now exist.

Lock file was not generated?

Confirm that the source file is saved at .github/workflows/ai-test-guard.md.
Confirm that the opening frontmatter has matching --- delimiters.
Ask for help diagnosing the compilation failure.

Publish the source and generated workflow files on main by running these commands:

git add .github/workflows/ai-test-guard.md .github/workflows/ai-test-guard.lock.yml
git commit -m "Add AI Test-Guard workflow"
git push

What does this publish?

The first command stages the editable source and compiler output.
The second command records both files in one commit.
The final command publishes the commit to the public repository's main branch.

The main branch now contains the advisory workflow that reviews newly opened pull requests and synchronized updates.

Workflow push failed?

Confirm that GitHub CLI remains authenticated to the repository owner account.
Confirm that both workflow files were included in the commit.
Ask for help with the workflow push.

Open the pull request

The same feature diff can now reach two reviewers. Deterministic CI checks its existing assertions while AI Test-Guard evaluates whether the changed behavior is represented in those assertions.

Return to the existing feature/infrastructure-risk branch by running this command:

git switch feature/infrastructure-risk

Why return to this branch?

This branch contains the committed infrastructure/* behavior that has no matching test. Its unpushed commit becomes the pull-request diff.

Publish the feature branch with an upstream reference by running this command:

git push -u origin feature/infrastructure-risk

What does the upstream reference do?

The push publishes the branch to GitHub. The upstream reference connects later pushes from this local branch to its remote counterpart.

Create the pull request against main by running this command:

gh pr create --base main --title "Add infrastructure risk classification" --body "Adds high-risk classification for infrastructure changes."

What does this create?

GitHub opens a pull request from the current feature branch into main. The command prints its URL when creation succeeds.

Before you watch the checks, which reviewer do you think can notice a new behavior with no matching assertion?

Watch the pull request checks until they finish by running this command:

gh pr checks --watch

What does the split verdict show?

The deterministic CI check stays green because its two known assertions still pass. AI Test-Guard identifies that infrastructure/* returns high without a matching assertion.

That split verdict is the intended result. Passing known tests does not prove that every changed behavior has a test.

Open the pull-request URL printed by the creation command in your browser.
Read the AI Test-Guard review to identify the uncovered behavior.

AI review is missing?

Confirm that the pull request targets main.
Confirm that the compiled workflow exists on the main branch.
Confirm that the repository secret is named GEMINI_API_KEY.
Ask for help diagnosing the missing review.

Add the missing coverage

The AI review has identified a specific test gap. A matching assertion now gives deterministic CI direct evidence for the new infrastructure-risk behavior.

Return to tests/deployment-risk.test.sh in your code editor.
Find this block at the bottom of the file:

assert_equal "low" "$(classify_path "docs/runbook.md")" "documentation risk"
assert_equal "medium" "$(classify_path "src/api.sh")" "application source risk"

printf '2 tests passed\n'

Where is the gap?

The current suite checks documentation paths and application source paths. It has no assertion for infrastructure/network.tf.

Replace that bottom block with this updated version:

assert_equal "low" "$(classify_path "docs/runbook.md")" "documentation risk"
assert_equal "medium" "$(classify_path "src/api.sh")" "application source risk"
assert_equal "high" "$(classify_path "infrastructure/network.tf")" "infrastructure risk"

printf '3 tests passed\n'

What changed in the suite?

The new assertion passes infrastructure/network.tf into classify_path(). It requires the classifier to return high.

The final message now reports three passing tests. That count makes the added coverage visible in local and hosted runs.

Save tests/deployment-risk.test.sh.
Compare the completed test file with the reference below.

✔️ Awesome, I've got everything!

Your test file now contains the infrastructure-risk assertion. Keep the file saved before running it.

ⓧ I'd like to double check the full code

#!/usr/bin/env bash
set -euo pipefail

source "$(dirname "$0")/../scripts/deployment-risk.sh"

assert_equal() {
  local expected="$1"
  local actual="$2"
  local description="$3"

  if [[ "$actual" != "$expected" ]]; then
    printf 'FAIL: %s. Expected %s, got %s\n' "$description" "$expected" "$actual" >&2
    exit 1
  fi
}

assert_equal "low" "$(classify_path "docs/runbook.md")" "documentation risk"
assert_equal "medium" "$(classify_path "src/api.sh")" "application source risk"
assert_equal "high" "$(classify_path "infrastructure/network.tf")" "infrastructure risk"

printf '3 tests passed\n'

What should match?

Compare all three assertions with your saved file. The final output line must report 3 tests passed.

Before you run the suite, do you expect the new assertion to pass against the classifier already on this branch?

Run the updated deterministic suite by running this command:

bash tests/deployment-risk.test.sh

What should the test report?

The terminal prints 3 tests passed. The new behavior now has a deterministic assertion.

Tests did not reach three?

Confirm that the new assertion uses infrastructure/network.tf as its input.
Confirm that the expected value is high.
Confirm that the final output line reports three tests.
Ask for help debugging the failing assertion.

Publish the test fix to the existing pull request by running these commands:

git add tests/deployment-risk.test.sh
git commit -m "Test infrastructure risk classification"
git push

How does the pull request update?

The commit records the new assertion on feature/infrastructure-risk. The push synchronizes the existing pull request.

That synchronization triggers deterministic CI and AI Test-Guard again. Both reviewers now receive the updated diff.

Before you run the final check, do you expect deterministic CI and AI Test-Guard to agree this time?

Watch the synchronized pull request checks by running this command:

gh pr checks --watch

What confirms the guard works?

The deterministic CI check passes with all three assertions. The new AI review states that the changed infrastructure behavior has a matching test.

The agent stayed inside its advisory boundary throughout the review. Your code changed only after you chose to respond to its finding.

Return to the existing pull-request URL in your browser.
Confirm that the latest AI review recognizes the matching infrastructure-risk test.

That closes the loop. Your deterministic pipeline now proves known behavior while AI Test-Guard checks whether the pull-request diff introduced an uncovered branch.

Secret mission

Turn Advice into a Review Gate

Upgrade AI Test-Guard so uncovered behavior receives a bounded REQUEST_CHANGES review. Then prove deterministic CI can stay green while the review gate blocks an incomplete change.

Clean Up Your Resources

This project should stay at $0 with a public ai-test-guard repository and Free Tier Gemini usage. Decide whether to keep everything active, pause AI review, or delete the resources.

Resources you used:

Your public GitHub repository named ai-test-guard. This includes the open pull request, branches, workflows, and GitHub Actions run logs.
Your Google AI Studio auth API key. The repository stores it as the Actions secret GEMINI_API_KEY.
Your local ai-test-guard directory and its project files.
Your local GitHub Agentic Workflows compiler installation at gh-aw v0.88.4.

Keep everything running

No action needed. Choose this if you want to keep testing the review gate with future pull request updates.

Keep ai-test-guard public so standard GitHub-hosted runner use stays free.
Leave feature/infrastructure-risk open and unmerged because its secrets/* behavior intentionally lacks a test.
Retain GEMINI_API_KEY in the repository Actions secrets if you want AI Test-Guard to review future pull request updates.
Keep the repository synthetic because Free Tier Gemini prompts and responses may be used to improve Google products.

Pause - I'll come back to this later

Pause AI review while keeping the repository files and deterministic CI available. The local compiler has no running process to stop.

Remove GEMINI_API_KEY from the ai-test-guard repository's Actions secrets settings.
Keep .github/workflows/ci.yml enabled so deterministic tests continue running.
Leave the intentionally incomplete pull request unmerged.

That gives you a clean pause. Deterministic CI stays available without an active AI credential.

Delete - I don't want to use this again

Remove every project resource if you want a full reset. Deletion permanently removes the open pull request and its review history.

Save any review text or project files that you need later.
Remove GEMINI_API_KEY from the ai-test-guard repository's Actions secrets settings.
Revoke the corresponding auth API key from the API key management page in Google AI Studio.
Permanently delete the public ai-test-guard repository from the repository deletion area in its settings.

Deleting the repository also removes its branches, workflows, Actions logs, open pull request, and stored repository secrets.

Move the local ai-test-guard directory to Trash in Finder.
Empty Trash to remove the local project files permanently.
Remove the local gh-aw extension by running this command:

gh extension remove gh-aw

What does this command do?

The command removes the gh-aw extension from your authenticated GitHub CLI installation.

The next check is expected to stop finding an installed gh-aw version.

Confirm the compiler extension is gone by running this command:

gh aw version

What should I see?

The gh aw version command should no longer report an installed gh-aw version.

Still seeing a gh-aw version?

Confirm that the extension name in the removal command is gh-aw. Run the removal command again if the earlier command did not complete.

Help me remove the gh-aw extension from GitHub CLI.

That completes the teardown. Your repository, credential, local project files, and compiler extension are removed.

Nice Work!

You did it! You built AI Test-Guard, a pull-request reviewer powered by deterministic Bash tests plus bounded AI judgment.

Your open pull request now tells the full story. Conventional CI stays green while the bounded reviewer catches missing secrets/* coverage for a human to resolve.

What you learned:

Build a deployment-risk classifier that returns low, medium, high, critical, or unknown for changed paths.
Run deterministic pull-request tests with GitHub Actions on every push or pull request.
Use a Gemini-powered review workflow to find behavior without a matching assertion. Respond to its finding with the missing infrastructure test.
Secret Mission: Convert advisory feedback into a bounded REQUEST_CHANGES review gate that catches an untested secrets/* branch. Keep the incomplete commit unmerged for a human decision.

Ready to quiz yourself?