Skip to main content

​
Scenarios

An API scenario is a reusable test for one API behavior. It describes the request steps to run, the data to capture, the assertions to check, and whether the scenario is still a draft or ready for scheduled runs.

​
How a scenario is structured

Each scenario is stored on the project with the steps, captures, and assertions needed to test one behavior. Here is an abbreviated shape: 
{
  "id": "scn_a1b2",
  "name": "POST /users returns 201 with id",
  "description": "Happy path: valid payload, valid auth, expect 201 + id in body.",
  "type": "api",
  "status": "draft",
  "priority": "high",
  "tags": ["users", "happy-path"],
  "steps": [
    {
      "order": 1,
      "kind": "api",
      "action": "create user",
      "target": "${API_BASE_URL}/users",
      "method": "POST",
      "auth": { "type": "bearer", "token": "${AUTH_TOKEN}" },
      "body": { "email": "test+${TS}@example.com", "name": "Ada" },
      "expectedStatus": 201,
      "captures": [{ "as": "userId", "from": "$.id" }],
      "expectations": [
        { "kind": "status", "equals": 201 },
        { "kind": "jsonpath", "path": "$.id", "exists": true }
      ]
    }
  ]
}
Important fields:
  • type is api, ui, or mixed. This page covers api.
  • status is draft or active. The scheduler only runs active.
  • steps[] are ordered. Each step has an action, a target, optional auth, optional body, optional captures for chaining, and expectations (the assertions).
  • tags filter scenarios at run time (smoke, checkout, auth).
  • priority is critical, high, medium, or low.

​
Lifecycle: draft to active

Every scenario starts as draft. Qodex can recommend and generate tests, but humans decide when a test is ready for automation.
StatusWho can run itUsed for
draftYou, manuallyReview, edit, promote
activeScheduler, webhooks, CI, youRegression runs
Promote a scenario in the UI when the steps and assertions look right. Demote back to draft if the underlying endpoint changes and you need a quiet window to update. The draft default is deliberate. Generated scenarios are recommendations, not releases. Scheduled suites only execute scenarios you have explicitly promoted.

​
Authoring from chat

In a chat, describe the behavior: 
Test that POST /orders requires auth: no token -> 401,
expired token -> 401, valid admin token -> 201.
Qodex picks the matching endpoints from the catalog, generates one scenario per behavior, and verifies them against your default staging environment. The verdict (pass, fail, error) attaches to the scenario row immediately so you can triage right away. Prompts that work well:
  • Name a behavior, not a step list. “Test the refund flow” beats “POST /refunds then GET /orders/123”.
  • Reference your terms: module names (auth, users, billing), HTTP codes (401, 403, 422), and edge cases (missing email, expired token).
  • Constrain auth explicitly when it matters: “as admin”, “without a token”, “as a viewer trying to write”.

​
Authoring manually

Skip the agent when you already know exactly what request and assertions you want.
  1. Open Scenarios in the sidebar.
  2. Click New scenario.
  3. Pick the endpoint from the catalog, or type a URL directly.
  4. Fill in params, headers, body, auth on the step.
  5. Add captures if a later step needs a value from this one.
  6. Add expectations: status codes, JSONPath checks, response-body shape.
  7. Save.
Manual scenarios skip the LLM authoring loop. Auto-verification still runs on save.

​
Combining AI and manual edits

You can edit any agent-generated scenario by hand: change the body, add a header, swap auth, or tighten an assertion. The next time you ask the agent to extend the scenario, it reads the current shape and edits on top instead of regenerating from scratch. Common pattern: let the agent scaffold the structure, including steps, captures, and basic expectations. Then hand-edit the values and assertions that depend on your business rules.

​
Auto-verification

API scenarios run against the target environment the moment you save them. The verdict attaches to the row immediately. See Auto-verification on save for what is checked and what is not.

​
On the roadmap

Self-critique on save (intelligence track): a flagship LLM reviewer reads every generated scenario against its stated goal and endpoint context, returns a structured verdict (approve, revise, reject) and a strength score (0 to 100), and flags weak scenarios with issue lists in the scenarios sidebar. Advisory only, never blocks the save. The reviewer is a different role from the coordinator: plan before, judge after.

Chaining and postscripts

Reference earlier-step outputs in later steps.

Test rules in plain English

Assertions written in English, executed as JavaScript.

Request data generation

How Qodex fills params, headers, and bodies.

Auto-verification on save

The verdict that attaches the moment you save.