Evaluating CodeRabbit? Same review, plus real test runs. See why

API testing

API tests written from your spec. Run on every pull request.

Import a spec, a Postman collection, a spreadsheet, or one sentence. Runnable scenarios, run against every PR preview, replayed for $0.

Two weeks → two days

to ship a change

Mavim’s APIs sit behind Microsoft SSO and browser-based authentication. Qodex runs their end-to-end API scenarios through that flow, so a change that took a couple of weeks to ship now ships in two days.

Wouter Jan de BaanCPO, Mavim
Read the Mavim case study
The suite

How Qodex tests your API

Intelligence is spent once, at authoring. Every replay after that is Playwright and HTTP code with no model in the loop, so the suite grows without the bill growing.

Product view. Example data.

Playground

qodex · playground · preview-1473
Agentcheckout-api
Test that checkout rejects an expired card and preserves the cart after auth refresh.
Sub-agent: explore checkout decline pathCompleted
Phase 1 · reproduce the decline on the real checkout.
Opened /checkout
200 · cart has 2 items, total $48.00
Filled card 4000 0000 0000 0069
expiry 04/22 · cvc 123 · submitted
Phase 2 · assert the gateway refuses it cleanly.
POST /v1/payments -> 402
{ "error": "card_expired", "decline_code": "expired_card" }
Asserted decline copy "Your card has expired"
visible on #pay-error · charge refused
Phase 3 · confirm the cart survives a session token refresh.
POST /v1/auth/refresh -> 200
rotated session token · same cart id
GET /v1/cart -> 200 ...
2 items preserved · verifying line totals
Exploring checkout endpoints6/12 mapped
ScenarioTS-042

Expired card is declined cleanly

POST /v1/payments · checkout-api
Severity
HIGH
State
ACTIVE
Last run
PASS
Steps
  1. 1.
    GIVEN
    A cart with two items at /checkout
    NAVIGATE /checkout · total $48.00
  2. 2.
    WHEN
    Pay with an expired card
    POST /v1/payments · card 4000…0069 · expect 402
  3. 3.
    THEN
    Decline copy shows and the charge is refused
    ASSERT text="Your card has expired"
  4. 4.
    AND
    Cart survives a session token refresh
    GET /v1/cart -> 200 · 2 items preserved
HIGHPASS
run_5f1c · 4/4 assertions · 1.2ssaved to suite (42)
  • $0per replay
  • 3failure classes
  • 200scenarios in 48 hours
  1. Step 01

    Import

    Start from an OpenAPI spec, a Postman collection, a spreadsheet, or a chat brief: describe the flow in a sentence and the agent drives the app.

  2. Step 02

    Generate

    Runnable scenarios, auth and role boundaries included, plus contracts and performance budgets from the same agent.

  3. Step 03

    Run on every PR

    The suite runs against the pull request’s own preview, and on demand, on a schedule, from CI, a deploy hook, or any webhook.

  4. Step 04

    Evidence

    Every failure comes back with the failing request, the response, and a screenshot. Nothing to reproduce by hand.

  5. Step 05

    Triage

    Each failure is classified: a real bug, a stale test with the repair proposed as a diff you approve, or an environment issue flagged and not counted.

  6. Step 06

    Replay for free

    A replay is Playwright and HTTP with no model call. A nightly full-suite run is a checkbox, not a budget line.

The GitHub app installs in minutes. Nothing goes into your application and no runner goes into your CI. The next pull request gets tested.

What it catches

One example run.

One scenario from a checkout suite: the request it sent, the response it got, and what the run did about it.

FailedAnonymous user cannot create an order
checkout · regression suite
Request
POST /api/orders
Authorization: (none)
{ "sku": "PRO_77", "qty": 1 }
Response
201 Created
{ "id": "ord_8f21", "status": "pending" }
expected 401 · got 201request, response and screenshot attached
Real bug · filed
Coverage · pages and endpoints48 total
  • 41 tested
  • 6 untested
  • 1 failing

Example data. A real run from a public API replaces this.

Your team

Built for the way your team tests

  • Tests you own

    Standard Playwright and HTTP code. Read it, edit it, export it, keep it if you leave.

  • Parameterized per environment

    One scenario, run against local, staging, the preview, or production, with the values swapped in.

  • Git-synced

    The suite lives next to the code it tests and moves with the branch.

  • A coverage map you can see

    Every page and endpoint marked tested, untested, or failing. The gaps are the next brief.

  • Run from anywhere

    On demand, on a schedule, from CI, from a deploy hook, or from any webhook.

Your stack

Works with what you have

The spec you wrote. The collection you kept. The CI you run. Qodex tests it.

  • OpenAPI
  • Postman
  • Playwright
  • GitHub
  • GitLab
  • CLI
  • Slack
  • Webhooks
Add-on

And it reviews the pull request too.

The same agent that runs your API scenarios reads the pull request: static analyzers, a full read of every changed file, a blast-radius pass, two frontier models, and live probes against the preview. Test failures and review findings land in the same Check Run.

See AI code review
Review finding
qodexroutes/orders.ts line 58

The auth middleware was dropped from the orders router in this change, so POST /api/orders no longer requires a session.

verified on preview
Test run
POST /api/orders
as anonymous
expected 401 · got 201

Screenshot attached · scenario: Anonymous user cannot create an order

Qodex / api · 1 real bug · merge blocked
evidence

Example. Real ones link to the pull request.

Security

Built to be trusted with your API.

  • Read access only

    The GitHub app asks for read access. Nothing is installed in your application and no runner goes into your CI.

  • Encrypted and isolated

    Data is encrypted in transit and at rest, and isolated per project.

  • Never trains on your code

    Your code, specs, scenarios and run history are never used to train models.

  • Bring your own key

    Use your own model key if you prefer. Authoring is the only step that spends it.

Customers

What teams say on G2.

  • G2 · Enterprise
    We now detect issues in staging instead of finding them after deployment.

    Anshuk K.

    AI & Data Engineer

    Verified G2 review
  • G2 · Financial services
    The no-code interface helps our whole integration team write and manage test cases, and drastically reduces the time spent on QA.

    Rishabh V.

    AI Engineer

    Verified G2 review
  • G2 · Small business
    It lets our small team stay confident in releases without needing a dedicated QA resource.

    Sameer C.

    Software Engineer

    Verified G2 review
FAQ

Questions teams ask before switching.

What can Qodex generate API tests from?

An OpenAPI spec, a Postman collection, a spreadsheet of endpoints, or a chat brief. Describe the flow in a sentence and the agent drives the app, captures the requests, and brings back reproductions. The output is the same in every case: runnable scenarios with contracts, auth and role boundaries, and performance budgets from one agent.

Does it work with APIs behind SSO or browser-based login?

Yes. Mavim’s APIs sit behind Microsoft SSO and browser-based authentication; Qodex runs their end-to-end API scenarios through that flow. Scenarios carry the roles they run as, so an admin, a member, and an anonymous caller are tested against the same endpoint.

How does it handle flaky and stale tests?

Every failure is classified before it reaches you. A real bug arrives with the failing request, the response, and a screenshot. A stale test, where the API changed and the expectation no longer matches, arrives with the repair proposed as a diff you approve. An environment issue such as the target being down is flagged and not counted, so a scheduled suite stays trustworthy instead of noisy.

Do we own the tests?

Yes. Generated tests are standard Playwright and HTTP code, parameterized per environment, synced to git, and exportable. There is no proprietary runtime to learn and nothing that stops working if you stop paying.

What does a run cost?

Intelligence is spent once, at authoring. A replay is Playwright and HTTP with no model call, so it costs $0 in model spend whether the suite has ten scenarios or a thousand. A nightly full-suite run is a checkbox, not a budget line. Bring your own model key if you want authoring on your own bill.

How is this different from Postman collections?

A Postman collection is requests you built and assertions you wrote, run by hand or by the collection runner. Qodex starts from that collection or your spec, writes the scenarios, runs them against the preview on every pull request, and classifies every failure. Keep Postman for ad-hoc requests; the regression suite moves to Qodex and becomes code you own.

Something else? Get in touch

Get started

Stop merging on faith.

Install the GitHub app, point Qodex at your staging URL, and the next pull request gets tested before it merges.

  • Nothing to install in your app
  • No credit card
  • Scenarios you own