Auditing Agent-Authored PRs Before Merge

A human PR author has self-correcting friction. Adding a new dependency means typing the package name, watching npm install run, dealing with the lockfile diff. They notice. They might google the package first. An agent doesn't have that friction — it adds the dependency because the prompt suggested a feature, and the package name appears in the diff three seconds after the operator hits enter.

The same asymmetry applies to: pasted secrets (the operator may not see the secret in the agent's output), license-incompatible code (the agent doesn't read GPL preambles), accidental schema rewrites (the agent decided to "improve" the migration), and bulk file edits (the agent changed 80 files when the prompt only suggested changing one).

Most of these failure modes are detectable with cheap, deterministic checks that run pre-merge. This article enumerates the merge-time gates we recommend for agent-authored PRs and gives you the GitHub Actions workflow to enforce them.

The Heuristic-First Strategy

Before diving into specific scanners, the meta-pattern: gate on diff shape first, content second. A small focused diff (5 files, 80 lines, no new dependencies) gets light scrutiny. A large multi-system diff (40 files, 2,400 lines, 3 new dependencies, schema migrations included) gets the full battery. The shape of the diff is a strong signal of whether the agent ran outside the operator's intent — and it's free to compute.

The shape-based heuristics that warrant escalation:

These are not pass/fail gates — they're escalation triggers. A PR that hits 3+ triggers gets routed to a senior reviewer and requires the full content-scan battery. A PR that hits zero triggers can fast-track through standard review.

The Diff-Shape Workflow

# .github/workflows/agent-pr-shape-gate.yml
name: Agent PR Shape Gate

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  shape-check:
    if: contains(github.event.pull_request.labels.*.name, 'agent:cursor') || contains(github.event.pull_request.labels.*.name, 'agent:copilot') || contains(github.event.pull_request.labels.*.name, 'agent:claude-code')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Compute diff shape
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: |
          FILES_CHANGED=$(git diff --name-only "$BASE_SHA".."$HEAD_SHA" | wc -l)
          LINES_ADDED=$(git diff --shortstat "$BASE_SHA".."$HEAD_SHA" \
            | grep -oP "\d+(?= insertion)" || echo 0)
          MIGRATIONS_TOUCHED=$(git diff --name-only "$BASE_SHA".."$HEAD_SHA" \
            | grep -c "^migrations/" || true)
          CI_TOUCHED=$(git diff --name-only "$BASE_SHA".."$HEAD_SHA" \
            | grep -c "^\.github/" || true)
          DOCKER_TOUCHED=$(git diff --name-only "$BASE_SHA".."$HEAD_SHA" \
            | grep -ciE "(^|/)Dockerfile|\.dockerfile$" || true)

          # Detect new top-level dependencies in package.json / requirements.txt / go.mod
          NEW_DEPS=0
          if git diff "$BASE_SHA".."$HEAD_SHA" -- package.json \
            | grep -E "^\+\s+\"[a-z@]" | grep -v "^\+\+\+"; then
            NEW_DEPS=$((NEW_DEPS + 1))
          fi
          if git diff "$BASE_SHA".."$HEAD_SHA" -- requirements.txt \
            | grep -E "^\+[a-zA-Z]" | grep -v "^\+\+\+"; then
            NEW_DEPS=$((NEW_DEPS + 1))
          fi

          TRIGGERS=0
          [ "$FILES_CHANGED" -gt 20 ] && TRIGGERS=$((TRIGGERS + 1))
          [ "$LINES_ADDED" -gt 500 ] && TRIGGERS=$((TRIGGERS + 1))
          [ "$NEW_DEPS" -gt 0 ] && TRIGGERS=$((TRIGGERS + 1))
          [ "$MIGRATIONS_TOUCHED" -gt 0 ] && TRIGGERS=$((TRIGGERS + 1))
          [ "$CI_TOUCHED" -gt 0 ] && TRIGGERS=$((TRIGGERS + 1))
          [ "$DOCKER_TOUCHED" -gt 0 ] && TRIGGERS=$((TRIGGERS + 1))

          echo "Diff shape: files=$FILES_CHANGED lines=$LINES_ADDED deps=$NEW_DEPS"
          echo "Sensitive paths: migrations=$MIGRATIONS_TOUCHED ci=$CI_TOUCHED docker=$DOCKER_TOUCHED"
          echo "Triggers: $TRIGGERS"

          if [ "$TRIGGERS" -ge 3 ]; then
            gh pr edit "$PR_NUMBER" --add-label "agent:high-risk"
            gh pr comment "$PR_NUMBER" --body "Agent PR escalated: $TRIGGERS shape triggers fired. Senior reviewer required."
          elif [ "$TRIGGERS" -ge 1 ]; then
            gh pr edit "$PR_NUMBER" --add-label "agent:elevated-review"
          fi

Note the labels: agent:elevated-review and agent:high-risk flow into the reviewer assignment workflow (Reviewer Assignment When Half Your Team Is an Agent) and into your PR dashboard so reviewers can prioritise.

Secret Scanning: The Non-Negotiable

Agents leak secrets in three distinct ways, none of which a human author would typically reproduce:

1. Pasted-from-prompt secrets — the operator pasted an API key into the prompt as context; the agent helpfully echoed it into a config file.
2. Hallucinated example secrets that happen to be real — agents trained on public code sometimes regurgitate real keys from training data.
3. Test fixtures that copy real secrets — agent generates a test that needs a token, sees a real token in the dev .env, copies it into the test file.

The mitigation is the same for all three: a pre-merge secret scanner that fails the build on any high-confidence detection. We recommend Gitleaks or TruffleHog — both are open source, both have entropy-based detection on top of regex rules, and both run in seconds on a typical PR diff.

# In the same workflow
secret-scan:
  if: contains(github.event.pull_request.labels.*.name, 'agent:')
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Run gitleaks
      uses: gitleaks/gitleaks-action@v2
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        config-path: .gitleaks.toml

For agent PRs specifically, lower the entropy threshold and add the --no-git mode that scans the working tree (not just the diff), because some agents will revert a leak in a later commit while leaving the secret in git history.

Dependency Addition Detection

A new top-level dependency is one of the highest-leverage attack vectors in the modern supply chain. Typosquatting, dependency confusion, and compromised maintainer accounts all weaponise this. When the dependency is added by an agent, the human chain of responsibility is even thinner — the operator may not have heard of the package before the agent suggested it.

The control is straightforward: any agent PR that adds a new top-level dependency requires explicit human approval of that dependency choice, separate from the PR approval. Encode it as a label-based gate:

new-dep-gate:
  if: contains(github.event.pull_request.labels.*.name, 'agent:')
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Detect new dependencies
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        PR_NUMBER: ${{ github.event.pull_request.number }}
        BASE_SHA: ${{ github.event.pull_request.base.sha }}
        HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      run: |
        # Extract added top-level deps from package.json
        NEW_DEPS=$(git diff "$BASE_SHA".."$HEAD_SHA" -- package.json \
          | grep -E "^\+\s+\"[a-z@]" \
          | grep -v "^\+\+\+" \
          | sed -E 's/^\+\s+"([^"]+)".*/\1/' \
          | sort -u)

        if [ -n "$NEW_DEPS" ]; then
          COMMENT="Agent PR adds new top-level dependencies:\n\n"
          for DEP in $NEW_DEPS; do
            COMMENT+="- \`$DEP\`\n"
          done
          COMMENT+="\nReviewers: please verify each dependency is from a trusted publisher and required for this change."

          gh pr comment "$PR_NUMBER" --body "$COMMENT"
          gh pr edit "$PR_NUMBER" --add-label "agent:new-deps"

          # Require explicit approval label to merge
          echo "::warning::New dependencies require explicit approval; add 'deps-approved' label to merge"
          exit 1
        fi

Combine with branch protection: require either zero agent:new-deps PRs or a deps-approved label (granted only by a member of the supply-chain owners team).

License Scanning

Agents trained on GitHub will, with non-trivial probability, generate code that closely matches existing GPL or AGPL files. For commercial closed-source products, that's a license-compatibility nightmare. For projects that are themselves GPL (Chalk and Ocular, in our case), it's a different but equally important concern: ingesting non-GPL code in ways that fragment the license picture.

Run a license scanner against any new files in the PR diff. We recommend licensee for repository-level detection and scancode-toolkit for file-level scanning of generated code:

scancode --license --json-pp scan-results.json $(git diff --name-only --diff-filter=A "$BASE_SHA".."$HEAD_SHA")
jq '.files[] | select(.licenses | length > 0) | {path, licenses: [.licenses[].key]}' scan-results.json

Flag any GPL/AGPL/LGPL detection on a PR for a closed-source repo, or any non-GPL detection on a GPL repo. The scanner produces SPDX identifiers; the policy check is a one-line jq filter against your project's allowed license list.

Schema and Migration Diffs

Agents that touch migrations/ are particularly dangerous because a "fix" that drops a column can pass tests (no test exercises the dropped column) and silently break production data. The control is conservative: any agent PR that modifies migrations requires explicit DBA team review.

migration-gate:
  if: contains(github.event.pull_request.labels.*.name, 'agent:')
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Detect migration changes
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        PR_NUMBER: ${{ github.event.pull_request.number }}
        BASE_SHA: ${{ github.event.pull_request.base.sha }}
        HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      run: |
        MIGRATION_FILES=$(git diff --name-only "$BASE_SHA".."$HEAD_SHA" \
          | grep "^migrations/" || true)

        if [ -n "$MIGRATION_FILES" ]; then
          gh pr edit "$PR_NUMBER" --add-reviewer "@org/dba-team" --add-label "agent:schema-change"

          # Detect destructive operations
          if git diff "$BASE_SHA".."$HEAD_SHA" -- migrations/ \
            | grep -iE "DROP TABLE|DROP COLUMN|TRUNCATE|DELETE FROM"; then
            gh pr comment "$PR_NUMBER" --body "Destructive SQL detected in agent migration. DBA review required."
            gh pr edit "$PR_NUMBER" --add-label "agent:destructive-sql"
          fi
        fi

The Composite Merge Gate

All of the above checks should compose into a single required status check before merge. The check passes if and only if:

1. Diff-shape triggers are tracked and labelled
2. Secret scan returns clean
3. New dependencies are explicitly approved (or none added)
4. License scan returns no incompatible licenses
5. Schema changes have DBA approval (if applicable)
6. Human approval quorum from the routing workflow is satisfied

Configure branch protection to require this composite check before allowing merge to main. Agent PRs that meet all six can merge fast. Agent PRs that miss one block until the missing piece is resolved.

What This Looks Like Over a Quarter

Customers running this gate set typically see, in the first quarter after rollout:

• Agent PR merge rate drops from >95% to ~80% (the missing 15% are PRs that were going to cause incidents)
• Mean time-to-merge increases by ~40% on agent PRs (the gates take time)
• Production incidents traceable to agent-authored code drops by >60%
• Developer satisfaction with agent tooling increases (the gates make agent suggestions trustworthy enough to actually rely on)

The point is not to make agents harder to use. The point is to make the merge-time gate proportional to the failure modes that agents specifically introduce. Human PRs have decades of accumulated review intuition behind them. Agent PRs need that intuition encoded as deterministic checks until reviewers have caught up.

What to Build First

1. The diff-shape workflow. It's free, fast, and produces immediately useful labels.
2. Gitleaks on every agent PR. Non-negotiable.
3. The new-dependency comment + label gate. Requires no scanner — just a git diff and a comment.

Adopt those three this week. The license and schema gates can follow once you have visibility into how often the patterns fire.