Failure classification
Failure classification keeps a test suite useful. Qodex does not treat every red run as a product bug. It first decides whether the failure is a real bug, a stale test, or an environment issue.The three classes
| Class | Meaning | What Qodex does |
|---|---|---|
REAL_BUG | The application likely behaved incorrectly | Opens or updates a finding with severity and evidence. |
STALE_TEST | The test no longer matches the app | Marks the scenario stale and suggests a repair. |
ENVIRONMENT_ISSUE | The target environment, auth, or network failed | Reports the infra issue without filing a product bug. |
Why this matters
Without classification, a renamed button, a staging outage, and a real authentication regression all look the same. That creates noisy reports and missed bugs. With classification:- Selector drift becomes a stale-test task.
- Staging downtime becomes an environment note.
- Product regressions become findings.
Examples
| Failure | Likely class |
|---|---|
| Button moved and cached UI selector no longer matches | STALE_TEST |
| Staging API returned connection timeout for every scenario | ENVIRONMENT_ISSUE |
| Login accepts an invalid token and returns user data | REAL_BUG |
| API shape changed intentionally but the assertion was not updated | STALE_TEST |
Security note
For security scenarios, the classifier still runs first. If the failure is a real bug, security rules then interpret it with inverted semantics: fail means the app allowed the attack.Related
Severity model
See how real bugs get impact-ranked.
Triage workflow
Review what happens after a finding opens.
Re-run failed tests
Confirm fixes and flaky failures.
Scenarios
Understand the test unit that can go stale.