Skip to main content

​
Run tests

Once a scenario is saved, Qodex can replay it from the web app, chat, a schedule, a webhook, your CI pipeline, or a coding agent connected over MCP. The same test run view collects the result, step output, artifacts, and findings no matter where the run started. This gives teams one place to run smoke checks, nightly regressions, security checks, UI flows, and API suites against staging, production, preview, or any other named environment.

​
How running tests works

The run flow has four parts:
  • Choose a trigger such as manual run, chat, schedule, webhook, CI, or MCP.
  • Choose a scope such as one scenario, a test group, a tag-filtered set, failed scenarios, or the full active suite.
  • Choose an environment so the same scenario can run against staging, production, or a preview target.
  • Review results in the test run detail page, with failures labeled as a functional regression, a vulnerability, a test that needs repair, or an environment problem.
API scenarios replay as executable scripts. UI scenarios replay through Chromium and Playwright using their recorded steps and cached actions. Repeat runs do not need the agent to regenerate the scenario.

​
Pick how the run starts

From the web app

Start a run manually and watch results stream in the browser.

On a schedule

Run smoke, regression, or security checks on a recurring cadence.

Via webhook

Trigger a run from CI, deploy scripts, or any external system.

​
Pick what runs

Single scenario

Replay one scenario against a chosen environment.

Tag-filtered scenarios

Run every active scenario with tags such as smoke, regression, or security.

Test groups and folders

Run related scenarios as one sequential flow or parallel folder.

Full suite

Run every active scenario and scenario group in the project.

Re-run failed tests

Replay a previous run or isolate scenarios that failed most recently.

In CI

Gate merges and deploys with Qodex runs from your pipeline.

​
What results include

Each run writes to the same test run model:
  • Overall status and per-scenario status, with passed, failed, and skipped counts.
  • Step-level pass, fail, and skipped results.
  • API request and response evidence, with copy buttons and the full response on demand.
  • UI screenshots and failure artifacts.
  • A label on each failure saying what it was, such as Functional regression, Vulnerability present, Test needs repair, or Environment.
  • A plain-English failure summary when Qodex can explain why a run failed.
  • A report you can request by email and share outside the app.
  • A line saying who was notified, for example Email sent to 3 recipients or Webhook failed.
  • Findings created from confirmed product or security issues.
A run in which some scenarios were skipped (for example destructive scenarios on a read-only environment) shows a neutral SKIPPED status instead of green PASSED, so a skip is never reported as a pass.

​
Use reports after a run

After a run finishes, open the run detail page to review the result. Click Download report when you need to share evidence with a teammate, customer, or release owner; Qodex prepares the report and emails it to you. For failed runs, click View summary to see why the failed scenarios failed, then inspect the failed scenario, step evidence, request or response data, and screenshots. The summary is there to shorten triage; the artifacts remain the source of truth. To dig into one result, select it and use:
  • Ask in chat to open a new chat about what that result returned. It answers from the run’s stored evidence, including large responses, without calling your API or opening the app again. Available for passed, failed, and errored results.
  • Fix in chat to open a chat that repairs a failing scenario.

​
Where to go next

Run tests from the web app

Start with the easiest manual run path.

Run tests in CI

Add Qodex as a required check in your release workflow.

​
On the roadmap

SARIF and JUnit XML export and a standalone qodex CLI are tracked on the Qodex roadmap.