Code Review Automation Workflow

Objective

Teams adopting AI coding assistants generate code faster, but manual review capacity doesn’t scale with it — larger, more frequent pull requests hit the same review bottleneck, which can quietly erase the productivity gain the assistant was supposed to provide. This workflow sets up automated AI code review that runs on every pull request, applies the same scoped-review principles as a manual pass, and enforces tiered quality gates so critical issues block merge while minor ones just get flagged.

Who this is for

Development teams already using GitHub Actions (or a similar CI system) who want consistent review depth on every pull request, regardless of reviewer availability or time zone. Most valuable for teams that have noticed review turnaround slowing down as AI-assisted development increased their PR volume. Small solo projects benefit too, mainly as a first-pass check before self-review rather than a replacement for human judgment.

Prerequisites

• A GitHub repository with Actions enabled
• An API key for your chosen model provider (Anthropic, OpenAI, or similar), stored as a repository secret
• Existing static analysis already in place (ESLint, ruff, or your language’s equivalent) — this workflow adds semantic review on top, not instead of it
• Agreement within the team on what severity levels should actually block a merge versus just warn

Tools used

• GitHub Actions (the automation platform this workflow is built on)
• Your existing static analysis tool (ESLint, ruff, or equivalent)
• An LLM API for the semantic review pass — see our Code Review Prompt Template for the scoped-review prompt this workflow automates
• A PR-commenting mechanism (GitHub’s own API, or a tool like reviewdog for inline line-level annotations)

The workflow

Step 1 — Set the trigger and filter out noise

Trigger the workflow on pull_request events specifically: opened, synchronize, and reopened. Exclude draft PRs by filtering for ready_for_review state, and exclude dependency-bot PRs (Dependabot, Renovate) by author — reviewing an automated version bump with a full semantic pass wastes tokens on a PR that doesn’t need it.

Step 2 — Extract the diff and apply file filtering

Pull the changed files and their diffs, then filter out anything that doesn’t benefit from semantic review — generated code, vendored dependencies, lockfiles, and large data files. Reviewing these wastes budget and produces noise the model has no meaningful way to evaluate.

Step 3 — Run static analysis first

Run your existing linter or static analyzer before the AI pass, not instead of it. Static analysis catches deterministic issues (unused variables, style violations, known-pattern bugs) cheaply and reliably; the AI pass is for semantic issues static analysis can’t catch — logic errors, missing edge case handling, security patterns that depend on context. Running static analysis first also means the AI review can skip re-flagging what the linter already caught.

Step 4 — Run the scoped AI review

Send the filtered diff to your chosen model using a scoped review prompt — our Code Review Prompt Template is built for exactly this, requesting findings with a line number, severity, and specific fix rather than open-ended commentary. Request structured output (JSON or a consistent Markdown format) so the next step can parse it programmatically rather than needing a human to read free-form prose.

Step 5 — Sort findings into severity tiers

Group the structured findings into tiers with different consequences: security vulnerabilities block merge, performance regressions warn but don’t block, style issues post as suggestions, and architecture concerns get flagged for discussion rather than treated as a pass/fail check. This tiering is what keeps the automation useful instead of becoming an all-or-nothing gate that either blocks everything or gets disabled out of frustration.

Step 6 — Post results and enforce the gate

Post findings as PR comments — ideally inline, line-level annotations rather than one large comment block, since inline feedback maps directly onto the code it’s about. Set the workflow’s exit status to fail the check only for the blocking tier (typically security-critical findings), so the merge gate reflects your team’s actual risk tolerance rather than treating every finding as equally severe.

Inputs & outputs

Inputs: the pull request diff, your static analysis configuration, the scoped review prompt, and your team’s severity-tier definitions.

Outputs: inline PR comments with line number, severity, and suggested fix per finding; a pass/fail status check reflecting whether any blocking-tier findings were raised; and, optionally, a summary comment aggregating counts by severity for a quick read before diving into individual comments.

Automation

The GitHub Actions trigger configuration for Step 1 looks like this:

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]

jobs:
  ai-review:
    if: github.event.pull_request.draft == false && github.actor != 'dependabot[bot]'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write

From here, subsequent steps checkout the code, run your linter, extract the diff, call the model API with the scoped review prompt, parse the structured response, and post comments via the GitHub API — each as a separate step in the same job, so a failure at any stage is visible independently rather than buried inside one large script.

Optimization

• Filter files aggressively before sending anything to the model. Lockfiles, generated code, and vendored dependencies add token cost without adding review value.
• Cache the review prompt as a stable system prompt where your provider supports it. Since the scoped review instructions don’t change per PR, this is exactly the kind of content prompt caching is built for — see that guide for the mechanics and current provider pricing.
• Skip the AI pass entirely for trivial diffs. A one-line config change or a typo fix in a comment doesn’t need a full semantic review — set a minimum diff size threshold below which only static analysis runs.
• Review your false-positive rate periodically, not just once at setup. A review prompt that was well-calibrated at launch can drift out of sync with your codebase’s conventions as the codebase evolves — revisit the scope and severity definitions every few months.

Handling disagreement between the AI review and a human reviewer

This comes up more often than teams expect, particularly in the first month while severity definitions are still being calibrated. Two patterns are worth planning for explicitly rather than resolving ad hoc each time they happen.

The AI flags something as blocking that a human reviewer disagrees with. Give reviewers an explicit override path — a specific PR label or a maintainer-only merge permission that bypasses the automated gate — rather than leaving no option but to fight with the pipeline. Log every override, though; a severity tier that gets overridden frequently is a signal the tier definition needs adjusting, not that reviewers should keep working around it silently.

The AI misses something a human catches that arguably should have been flagged. This is normal and expected — the review prompt’s scope and the model’s judgment both have limits. Treat these as calibration signal: if the same category of issue gets missed repeatedly, that’s a concrete, specific addition to make to the scoped review prompt rather than a reason to distrust the automation generally.

Neither pattern means the automation is broken — both are the normal process of tuning a review pipeline to a specific codebase’s actual risk profile, which takes a few weeks of real PR volume to get right, not a one-time setup decision.

Rolling this out gradually

Turning on a new blocking CI check immediately, for the whole team, on day one is the most common way this kind of automation gets disabled within a month. A staged rollout avoids that.

Week 1-2: warn-only mode. Run the full pipeline, post all findings as comments, but don’t fail the status check for anything — even security-tier findings. This surfaces how noisy or accurate the pipeline actually is against your real codebase before anyone’s merge depends on it.

Week 3-4: enable blocking for the security tier only. By this point you should have a sense of the false-positive rate from the warn-only period. If security findings have been consistently accurate, promote that tier to blocking while everything else stays as suggestions.

Month two onward: reassess tier boundaries based on real data. Some teams eventually promote performance regressions to blocking too; others keep the gate narrow indefinitely and rely on the suggestion-tier comments purely as reviewer aids. Both are reasonable outcomes — the point of the staged rollout is reaching that decision with evidence instead of guessing at launch.

A worked example

A team merging roughly 15 PRs a week sets the blocking tier to security findings only, with everything else posted as non-blocking suggestions. In the first month, the automated pass catches two real issues static analysis missed entirely: a SQL query built with string concatenation instead of parameterization (blocked, fixed before merge), and a missing null check on an API response that would have caused a production crash under a specific race condition (flagged as major, fixed within the same PR after review). Style and architecture suggestions get posted too, but the team treats those as optional reading rather than blockers — roughly a third get addressed, the rest are judged not worth the change at review time. Total monthly API cost for the review pass, filtered to exclude generated files and lockfiles, lands under $40 for their PR volume.

Common mistakes in this workflow

• Making every finding a blocking check. This is the fastest way to get the automation disabled — if style suggestions block merge as hard as security findings, teams route around the check rather than deal with it.
• Skipping static analysis and relying on the LLM for everything. Static analysis is cheaper and more reliable for deterministic issues — using an LLM for what a linter already catches wastes budget and adds latency for no benefit.
• Not filtering out bot PRs and generated files. Reviewing a Dependabot version bump or a generated lockfile with a full semantic pass burns tokens on diffs that don’t need it.
• Treating this as a replacement for human review entirely. The automation catches a real, meaningful category of issues, but architectural judgment and product-context questions still need a human who understands why the code exists, not just what it does.

Team roles and responsibilities

Someone needs to own this pipeline once it’s running, not just set it up and walk away. In practice, three responsibilities need a clear owner, even on a small team where one person holds all three:

• Prompt and severity-tier maintenance. Whoever notices a pattern of false positives or missed issues needs a clear path to actually update the scoped review prompt or tier definitions, rather than that feedback disappearing into a chat message no one acts on.
• Override authority. Decide upfront who can bypass a blocking finding, and make sure that permission is scoped narrowly enough to matter — if everyone can override, the blocking tier isn’t really blocking anything.
• Cost monitoring. API spend on this pipeline scales with PR volume, which can grow unpredictably. Someone should be checking actual monthly cost against the estimate from setup, not discovering a budget surprise three months in.

Downloadable template

Copy this table to define your team’s severity tiers before setting up the workflow:

Severity	Example finding	Consequence

Frequently asked questions

Which model should I use for the AI review step?

Any current-generation model with strong coding performance works — see our AI Coding Assistant Comparison Tool for current options and pricing. The choice matters less than the scoping and severity-tier setup around it; a well-configured pipeline on a mid-tier model tends to outperform an unscoped, unfiltered pipeline on a flagship one.

How much does this actually cost to run?

It scales with PR volume and diff size, but file filtering and skipping trivial diffs both meaningfully reduce it. Teams running this at moderate PR volume with proper filtering commonly land well under $50 a month — run your own numbers with the AI Model Cost Calculator based on your actual PR volume and typical diff size.

What if the AI review produces false positives?

Expect some, particularly early on — this is normal and part of why only the blocking tier should actually gate merge. Track false-positive patterns over the first few weeks and refine the scoped prompt’s instructions or your severity definitions accordingly, rather than assuming the initial configuration is final.

Can I use this with GitLab CI instead of GitHub Actions?

Yes — the underlying pipeline logic (trigger, filter, static analysis, AI review, tiered gating, post results) is platform-agnostic. GitLab CI uses different YAML syntax and its own merge request API for posting comments, but the workflow structure transfers directly.

How do we handle a large, pre-existing codebase with no static analysis set up yet?

Set up static analysis first, scoped to new and changed code only rather than the entire existing codebase — retrofitting linting rules onto years of legacy code usually surfaces thousands of pre-existing violations that have nothing to do with the current PR. Once that’s stable, layer the AI review pass on top using the same new-code-only scope, then expand coverage gradually rather than trying to bring the whole codebase into compliance at once.

Should this replace our existing human code review process?

No — treat it as a first pass that runs before human review, catching mechanical and security issues consistently so human reviewers can focus more attention on architecture, product fit, and judgment calls the automation isn’t built to make.