​

Why doesn’t my LLM bill grow when I rerun the suite?

Qodex uses the LLM to create tests, understand changes, and recover from drift. It does not call the LLM for every step of every saved replay. That is why rerunning a saved suite should not make your model bill grow linearly with suite size.

​

API replay

Saved API scenarios replay as normal HTTP requests. The runner substitutes variables, sends the request, evaluates assertions, and records the result. No model call is needed on the stable replay path.

​

UI replay

Saved UI scenarios store browser actions that worked for each step. On replay, Qodex runs those cached actions directly in Chromium. If the UI changes and the cached action fails, Qodex may use one recovery call to resolve the original intent against the current page. Recovery is the exception, not the normal replay path.

​

When the LLM is used

Qodex uses the LLM for:

• Authoring new scenarios.
• Reading specs, pages, and project context.
• Exploring unknown API or UI surfaces.
• Recovering from drift.
• Explaining failures and findings.
• Reviewing generated scenarios before saving.

It is not used for stable API replay or stable UI replay.

​

Why the first run costs more

The first run usually includes discovery and authoring. Qodex has to understand the surface, decide what to test, create scenarios, resolve UI steps, and verify that saved tests work. Later runs reuse that work.

​

Next steps