Check Run and merge gating
Qodex posts a GitHub Check Run on every review so the PR’s checks panel shows the review status.
By default, the check is advisory. You can opt in to merge blocking when you want verified findings above a chosen severity to stop a PR from merging.
How the Check Run works
Each review has a Check Run lifecycle: posted as in_progress when the review starts, updated to completed with a conclusion when the review finishes. A re-review on the same head SHA updates the same Check Run rather than stacking new ones.
There are three possible conclusions: neutral, success, and failure. Only verified findings at or above the block_on_severity threshold flip the conclusion to failure, and only when the repo opts in via .qodex.yaml.
Lifecycle
- Start: the Check Run is posted against the head SHA as
Qodex review, in_progress, with a placeholder summary while the review runs.
- Finish: the same Check Run is updated to
completed, with the computed conclusion, a link to the walkthrough comment, a severity-count summary, and the list of blocking findings if any.
- Re-review: a subsequent push to a new head SHA creates a fresh Check Run. A re-review on the same head SHA (via
@qodex review) updates the existing one.
Conclusion policy
The conclusion is decided per review:
- If
.qodex.yaml has pr_review.block_pr_merge: false (the default), the conclusion is always neutral. This is advisory mode.
- If
block_pr_merge: true and any verified finding meets block_on_severity, the conclusion is failure. The merge is gated.
- Otherwise the conclusion is
success.
Two non-obvious rules apply.
- Only verified findings count toward the gate. An unverified finding never flips the conclusion to failure. Unverified means “the diff suggested the bug but Qodex could not confirm it”; blocking on those would destroy trust faster than missing a bug.
- Skipped or failed reviews are always neutral. Draft PRs,
.qodex.yaml opt-out, repo unlinked, LLM errors, GitHub errors. Qodex never gates a merge because its own infrastructure had a problem.
Conclusion meanings
| Conclusion | When it fires |
|---|
neutral | Advisory mode (block_pr_merge: false). Or the review was skipped (draft, opt-out, unlinked). Or it failed (LLM error, GitHub error). |
success | Gated mode and zero verified findings at or above block_on_severity. |
failure | Gated mode and at least one verified finding at or above block_on_severity. |
Title and summary
The Check Run title and summary describe the outcome.
failure: title Blocking: N findings. Summary names the count, the lowest blocking severity, and the path to allow the merge.success: title Clean review (zero findings) or No blocking findings (some findings, none verified at threshold).neutral advisory mode: title Advisory: N findings or Clean review (advisory).
The Check Run details panel renders a severity-count table, a blocking findings list when any, a link to the walkthrough comment, and a one-line gate policy footer such as Gate policy: blocking - threshold: critical.
How to opt in to merge blocking
Two settings on the repo opt you in.
1. Set block_pr_merge: true in .qodex.yaml
pr_review:
block_pr_merge: true
block_on_severity: critical
2. Require the Qodex review check in branch protection
In GitHub, open Settings > Branches > Branch protection rules > Edit. Under “Require status checks to pass before merging”, add Qodex review. Without this, the Check Run still posts but GitHub will not block the merge button.
Tune the threshold
block_on_severity defaults to critical. Raise the bar by setting it higher (no severity is above critical, so critical is the strictest). Lower the bar by setting major, minor, nitpick, or info. Only verified findings at or above this severity flip the gate.
pr_review:
block_pr_merge: true
block_on_severity: major # block on verified critical OR major findings
On the roadmap
A learning loop is on the roadmap that refines the gate by recording each finding’s resolution state. Findings marked false-positive on a previous PR will not contribute to the gate on a future PR with the same shape.
