Skip to main content

​
Operating modes

Qodex can run the same saved scenarios in three ways: on-demand, on a schedule, or from an external event.

​
The shared model

A scenario is the unit of work. You usually create scenarios through chat, then run them against any environment whenever you need them. The agent, scheduler, webhook runner, and PR review flow all use the same scenario store, scripts, and findings pipeline. You author a scenario once, then reuse it wherever it needs to run.

​
On-demand

Use on-demand mode when you want to ask Qodex for something directly. You talk to the agent in chat. The coordinator breaks down the brief, starts sub-agents, and streams tool calls and results back in real time over a WebSocket. This mode is best for authoring, exploratory runs, triage, and one-off “test this PR” requests. The agent updates project memory as it works, so the next session can reuse what was discovered.

​
Scheduled

Use scheduled mode when you want coverage to run without a person asking for it. Saved scenarios run on daily, weekly, or custom cron schedules. Each schedule chooses a target environment, a tag filter, and an optional priority filter. Only active scenarios run on a schedule; drafts are skipped. A scheduled run uses the deterministic replay path. API scenarios make HTTP calls. UI scenarios replay through the intent runner’s step cache, falling back to the LLM resolver only on a cache miss. Scheduled runs publish a test_run record with per-scenario pass, fail, and error breakdowns. Any real bugs become findings in the project.

​
Event-driven

Use event-driven mode when another system should decide when Qodex runs. External systems trigger runs through per-project API keys (qk_*). A POST /webhooks/trigger call from CI, a deploy hook, or a custom GitHub Action can run one scenario, a tag, or the full active suite. Each project can mint multiple keys. Keys are SHA-256 hashed, revocable, and use-counted. The same key gates the GitHub App’s per-PR review and any CI integration you write. For the GitHub App, push and pull_request webhooks drive the PR review flow: walkthrough comment, inline findings, preview-deploy checks, and a Check Run that can gate merges.

​
Which mode to choose

  • On-demand: authoring new scenarios, debugging failures, running ad-hoc explorations.
  • Scheduled: nightly regression, weekly security audit, continuous coverage as the app evolves.
  • Event-driven: CI gates, deploy verification, PR review, third-party agents driving Qodex as a component.

​
On the roadmap

Planned: autopilot direction. Qodex moves from “you ask, it runs” to “it watches your app and tells you what changed.” Reflection passes after each run propose new scenarios as drafts and surface gaps as inbox questions. See product.md vision.
Planned: Qodex as MCP server. External agents (Claude Code, Claude Desktop, ChatGPT, Cursor) drive Qodex through per-project API keys over Streamable HTTP. See backlog.md.

How Qodex works

Coordinator, sub-agents, scan types, cost model.

Scenarios

The atomic unit of testing.

Run tests on a schedule

Set up cron-based runs.

PR review

The event-driven mode in production.