> ## Documentation Index
> Fetch the complete documentation index at: https://qodex.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Auto-verification on save

> Qodex runs API scenarios when you save them so broken drafts are caught immediately.

# Auto-verification on save

Auto-verification runs an API scenario as soon as it is saved.

This gives you fast feedback while the scenario is still being authored, before it becomes part of a scheduled suite.

## How it works

When `scenario_save` runs, Qodex does not just persist the scenario. For API scenarios, it also runs the scenario's steps against the default `staging` environment, captures pass/fail per step, and writes `last_run_status` back to the scenario row.

The verdict shows up in the scenarios list as a pass/fail badge before you even close the create dialog.

This is deliberate. A generated scenario that fails its first run is often broken in a fixable way: a stale schema, a missing capture, or a misnamed environment variable. Surfacing the failure during authoring lets you or the agent correct it before the scenario becomes active.

## What auto-verify checks

For each step:

* **HTTP status codes**: the actual response status against `expectedStatus`.
* **Response shape**: each declared `expectations` entry evaluates against the live response. JSONPath checks, body-shape assertions, and header checks all run.
* **Captures**: any `captures` on a step are evaluated against the live response. If a capture fails (the JSONPath doesn't resolve), the step fails.

For the whole scenario:

* **Pass**: every step's expectations evaluate true and no step errors out.
* **Fail**: at least one step's assertions evaluate false.
* **Error**: a step couldn't run at all (network failure, malformed request, env not found).

## What auto-verify does not check

* **Semantic correctness.** Auto-verify only enforces the assertions you or the agent wrote into the scenario. If the assertion is weak (`status == 200`, no body check), auto-verify will pass on bad data that returns 200.
* **Business invariants.** "The new user actually shows up in the admin list" is a multi-step check you have to explicitly write. Auto-verify won't infer it.
* **Side effects on other endpoints.** Auto-verify isolates per scenario. If creating a user breaks the auth flow for everyone else, you'll catch that in the next full-suite run, not in this save.

Auto-verify is a tight feedback loop, not a coverage guarantee.

## What you see in the UI

The scenario list row shows a verdict pill the moment the save completes:

```
scn_a1b2  POST /users returns 201 with id   [PASS]   Edited 3s ago
scn_c3d4  GET /orders/:id IDOR returns 403  [FAIL]   Edited 12s ago
```

Click into a `FAIL` row to see which step failed, the actual HTTP status, the response body, and the assertion that did not match. From there you usually:

* Edit the assertion if your expectation was wrong.
* Edit the request if you sent the wrong payload.
* Demote to `draft` and ask the agent to investigate.

## When it doesn't run

* **No staging environment.** Auto-verify needs a default `staging` environment to fire against. If you haven't created one, the scenario saves and gets `last_run_status: null`.
* **UI scenarios.** UI scenarios go through `scenario_record`, not `scenario_save`. They have their own post-author verify path (see the UI testing docs).
* **Scenarios in a group with assumed captures.** Group members that depend on an earlier member's captures get auto-verified at the group level once you wire the group up.

## On the roadmap

<Tip>
  Self-critique on save runs a second LLM pass in addition to auto-verify: it reviews the generated scenario against the stated goal and returns a structured verdict (approve / revise / reject) with an issue list. Auto-verify checks "does it run"; self-critique checks "does it test what it claims to test." The two run independently and both attach to the row.
</Tip>

## Related

<CardGroup cols={2}>
  <Card title="Scenarios" icon="list-checks" href="/api-testing-scenarios">
    The shape and lifecycle.
  </Card>

  <Card title="Test rules in plain English" icon="check-check" href="/api-testing-test-rules">
    How assertions get into a scenario.
  </Card>

  <Card title="Failure classification" icon="bug" href="/findings">
    What happens when an active scenario fails later.
  </Card>

  <Card title="Run tests" icon="play" href="/run-tests">
    Triggering scenarios on demand, on a schedule, or via webhook.
  </Card>
</CardGroup>