Skip to main content

Limits and caveats

Qodex’s PR review is designed for high-signal feedback on normal pull requests. These limits explain what it reviews today, what it skips, and how to work around common boundaries.

Quick summary

  • Large diffs are truncated at 200 KB.
  • Findings below 0.7 confidence are dropped.
  • Reviews are advisory by default unless you opt in to merge blocking.
  • Draft PRs and disabled repos are skipped with a neutral Check Run.
  • Verification probes are GET-only and require a discoverable preview deployment.

Diff truncation at 200 KB

The unified diff is capped at 200 KB before it is sent to the model. A PR that touches a large generated file, a vendored library, or a refactor across hundreds of files will be truncated. The agent is told the diff is truncated and is instructed to focus on what it can see. The walkthrough footer shows diff truncated at 200KB when this fires. What to do: split large PRs when possible. Qodex reviews each PR independently, so two 100 KB PRs get full review coverage where one 400 KB PR loses part of the diff.

Confidence floor at 0.7

Every finding has a confidence score between 0.7 and 1.0. Anything below 0.7 is dropped before it ever reaches your PR. The floor is not tunable per project today. What to know: this is intentional. Qodex optimizes for precision over recall so the PR thread stays useful.

Single LLM pass per review

Every finding comes from one call to the model. There is no second pass to catch what the first missed, no cross-finding consistency check, no multi-agent ensemble. The transparency block in the walkthrough explicitly says “single LLM pass, all dimensions”. What to know: a finding that requires looking at two files together may not surface if the diff hides one of them. Findings outside the diff are still bounded by what the diff reveals.

No scheduled re-review

A review fires on a GitHub event, not on a timer. If you change .qodex.yaml and want the new policy applied to an open PR, push a new commit or run @qodex review. The learning loop on the roadmap will not retroactively re-score past findings.

Draft PRs are skipped

A draft PR is skipped with a neutral Check Run and the reason “PR is a draft; Qodex will review on ready_for_review.” Until ready-for-review handling lands, push a new commit after marking the PR ready so the next event triggers a review.

Inline comments must land on lines in the diff

GitHub rejects inline comments on lines that are not part of the PR diff. Findings that reference a line the PR did not touch move into the walkthrough body under “Findings outside the diff” and are tagged advisory. The author still sees them.

Misanchor guard can strip a useful suggestion

When a finding is anchored on a comment, blank line, import, or brace-only line, Qodex strips the suggestion block and adds an Anchor uncertain badge. The check is conservative because a wrong suggestion can replace the wrong line. The description always still posts.

Check Run is advisory by default

.qodex.yaml ships with pr_review.block_pr_merge: false. The Check Run conclusion is always neutral until the repo opts in. Even when opted in, only verified findings at or above block_on_severity flip to failure; unverified findings never block. What to know: a fresh repo with no .qodex.yaml will never gate a merge. This is the safe default.

.qodex.yaml is capped at 16 KB

The config file is fetched from the PR head SHA and rejected if larger than 16 KB. A larger file falls back to defaults silently with a logged warning.

Repo size cap at 500 MB

A repo over 500 MB cannot be linked. The link POST returns 413 with the message “Repo is MB, exceeds 500MB cap. Subdirectory selection for monorepos is planned for v2.”

10 repos per project

A Qodex project can link at most 10 GitHub repos. Hitting the cap returns 409 with “Project already has 10 linked repos (max). Unlink one before adding another.” This cap is per project, not per install.

Probe is GET-only

Verification probes are GET-only today. Every other verb is rejected before any request is made. Non-idempotent probes are gated behind an explicit .qodex.yaml opt-in (probes.allow_non_get: true) and are on the roadmap.

Probe is one request

A probe is one GET. There is no multi-step probe, no login-then-probe flow, no probe chaining, and no probe retries. A flaky preview environment marks probes as failed. The finding still posts, just without the verification badge.

Preview deployment must come from GitHub Deployments

Qodex discovers the preview URL through the GitHub Deployments API. Vercel, Netlify, Cloudflare Pages, Render, and Railway all set Deployments by default. Custom CI that does not write to the Deployments API will not be discovered, and probes land as skipped.

On the roadmap

A learning loop is on the roadmap that adds resolution affordances and per-project filter training. Non-GET probes will open behind a .qodex.yaml opt-in. A future multi-agent review pipeline will lift the diff cap and the single-pass constraint.

Troubleshooting

.qodex.yaml reference

Verification probes

Check Run and merge gating