Evaluating CodeRabbit? Same review, plus real test runs. See why

Autopilot

Every production error becomes a bug that can never ship twice.

Qodex reads the failure, writes the regression as code you keep, proves it on staging, and fails any pull request that would bring the bug back.

Get startedBook a demo
It lets our small team stay confident in releases without needing a dedicated QA resource.
Sameer C.Software Engineer, verified G2 review
Read the review on G2
The loop

How the loop works

Autopilot is agentic AI for QA pointed at production. A person spots the bug once. From then on it is a scenario in your suite, replayed on every change with no model in the loop.

Product view. Example data.
Sentry · productionunresolved

TypeError: total is undefined

checkout/total.ts line 88312 usersfirst seen 2m ago
POST /checkout 500
promo: SPRING20 · expired 14:02:11
total = subtotal - promo.amount // undefined
logs · 14 linespayload attached
Saved scenarioverified on staging

Checkout applies an expired promo code

11 steps · playwright.spec.ts
  1. add item · apply promo SPRING20
  2. advance clock past promo expiry
  3. submit order
  4. expect total 49.00 · got 39.20
replay: no model callruns on every PR
  • 6stages, from error to merge gate
  • $0per replay, no model call
  • 1approval per scenario
  1. Stage 01

    A production error

    A real failure hits production and your monitoring catches it, whatever the source.

  2. Stage 02

    Qodex ingests it

    The error arrives with its logs and the real production data around it.

  3. Stage 03

    Understands the failure

    Reads it against the repo map it already holds: routes, auth, models, tests.

  4. Stage 04

    Writes the regression

    Authors a scenario that reproduces the bug, as standard code you keep.

  5. Stage 05

    Proves it on staging

    Runs the scenario against staging until the reproduction is verified.

  6. Stage 06

    Blocks the recurrence

    Any pull request that would reintroduce the bug now fails the merge gate.

Every pass through the loop deepens the per-project memory: auth flows, routes, models, past failures. Matchbook has run its production suite on Qodex since 2024, and the suite it runs today is the one production taught it.

The wider loop

Autopilot is stage six of the loop every pull request runs.

Memory feeds Change. The next pull request is tested against what production taught.

How it works

Every pull request, tested before it merges.

One agent, six steps, and the last one feeds the first.

  1. 1

    Change

    A developer or a coding agent opens a pull request.

  2. 2

    Context

    Qodex reads the diff and the repository around it.

  3. 3

    Execution

    It runs the relevant API, UI and security checks against the preview.

  4. 4

    Evidence

    Every finding carries the failing request, response or screenshot.

  5. 5

    Verdict

    The pull request gets a clear merge recommendation.

  6. 6

    Memory

    Production incidents become regression scenarios, so the next change is tested against them.

Back to 1. The next pull request is tested against what production taught.

No model call on replay. The suite grows; the bill does not.

What it produces

One example loop.

One incident from Sentry, the scenario it became, and the pull request that could not bring it back.

IncidentCheckout total mismatch
checkout/total.ts line 88

Orders with a promo code that expired mid-session were charged the discounted total. 312 users in 40 minutes before the alert fired.

Sentry · production
TypeError: total is undefined
Read from the bug your team filed
ScenarioCheckout applies an expired promo code
11 steps · playwright.spec.ts
01 add item · apply promo SPRING20
02 advance clock past promo expiry
03 submit order
   · · · 7 more steps
11 expect total 49.00 · got 39.20
Verified on staging · replay: no model call
Merge gatePR #1512 would reintroduce it
refactor: simplify promo validation

The refactor dropped the expiry check. The regression ran on the pull request and failed, so the merge is blocked until the check comes back.

qodex / regression · merge blocked

Example data. Real loops link to the incident and the pull request.

Roadmap, plainly

What is live today

The loop from repo map to merge gate runs now. The one piece still on the roadmap is named, not implied.

Live today
  • The repo map: routes, auth, models, tests
  • Regression authoring as standard Playwright and HTTP
  • Staging runs until the reproduction is verified
  • The merge gate on every pull request
  • Triggers from chat, CI, or any webhook
  • Bug ingestion from the tools your team files in: Sentry, Jira, Datadog, PagerDuty

Today a person approves each scenario before it joins the suite. That is one click, and it is the only human step in the loop.

On the integration roadmap
  • Automatic ingestion straight from your error monitor, with no one filing the bug

Until then the hand-off is the bug your team already files. Nothing else in the loop waits on it.

Your stack

Works with what you have

The monitor pages you. Your team files the bug. Qodex turns it into a regression.

  • Sentry
  • Jira
  • Datadog
  • PagerDuty
  • New Relic
  • GitHub
  • CI
  • Webhook
  • Chat

Read from where your team files bugs today. Automatic ingestion from the monitor itself is on the roadmap.

Add-on

And it reviews the pull request too.

The same agent that gates the merge on your regressions runs six review passes on the pull request itself: static analyzers, a full read of every changed file, a blast-radius pass, two frontier models, and live probes against the preview.

See AI code review
Autopilot on the pull request
qodex / regressionPR #1512

Checkout applies an expired promo code: failed. This pull request reintroduces incident 4471 from 12 June. Expiry check removed in promo/validate.ts.

regression · merge blocked
AI code review on the same pull request
promo/validate.ts line 31
High · expiry check removed, callers: checkout, cart, renewals
probed against preview · discount applied after expiry

Committable fix suggested on line 31 · found by both models

Qodex / review · 1 high · merge blocked
evidence

Example data. Real checks link to the pull request.

Security

Built to be trusted with production.

  • Read access only

    Nothing to install, no runner in your CI. Qodex asks for read access to the repo and the bug tracker.

  • Encrypted and isolated

    Incident data and scenarios are encrypted in transit and at rest, and isolated per project.

  • Never trains on your data

    Your code, incidents and scenarios are never used to train models.

  • Bring your own key

    Use your own model key for authoring if you prefer. Replay makes no model call at all.

Customers

What teams say on G2.

  • G2 · Enterprise
    “We now detect issues in staging instead of finding them after deployment.”

    Anshuk K.

    AI & Data Engineer

    Verified G2 review
  • G2 · Financial services
    “The no-code interface helps our whole integration team write and manage test cases, and drastically reduces the time spent on QA.”

    Rishabh V.

    AI Engineer

    Verified G2 review
  • G2 · Small business
    “It lets our small team stay confident in releases without needing a dedicated QA resource.”

    Sameer C.

    Software Engineer

    Verified G2 review
FAQ

Questions teams ask before turning it on.

What triggers the loop today?

A bug your team files. Qodex reads it from Sentry, Jira, Datadog or PagerDuty, or you hand it over from chat, CI, or any webhook. It reads the failure against the repo map, writes the regression, proves it on staging, and adds it to the merge gate. Automatic ingestion straight from the error monitor, with no one filing the bug, is on the integration roadmap.

Do we approve every scenario?

Yes. Each scenario is shown to you with the reproduction evidence from staging, and one approval adds it to the suite. That is the only human step in the loop, and it is the reason the suite stays one your team trusts rather than one it mutes.

What if the reproduction cannot be verified?

The scenario does not join the suite. Qodex keeps the draft with the runs it tried and what each returned, so a person can see why staging did not reproduce the failure: missing data, an environment gap, or a bug that only shows under production load. Nothing unverified gates a merge.

What do we own?

Everything the loop produces. Scenarios are standard Playwright and HTTP scripts, parameterized by environment variables, that you can read, edit and check into git. No proprietary runtime, no recording blob. If you leave, the tests leave with you.

What does a replay cost?

No model call. Authoring uses a model once; replay is plain code execution, the same requests and assertions every time. So the hundredth regression costs the same to rerun as the first, and running the whole suite on every pull request, on a schedule, from CI or from a webhook is an engineering decision rather than a budgeting one.

Which monitors are supported?

Today Qodex reads the bugs your team files in Sentry, Jira, Datadog and PagerDuty, and accepts any error over a webhook. Pulling errors automatically from the monitor itself, New Relic included, is on the integration roadmap. The rest of the loop, from repo map to merge gate, is live now and does not depend on it.

Something else? Get in touch

Get started

Stop merging on faith.

Install the GitHub app, point Qodex at your staging URL, and the next pull request gets tested before it merges.

Get startedBook a demo
  • Nothing to install in your app
  • No credit card
  • Scenarios you own