Skip to main content

Run tests in GitLab CI

Use a GitLab CI job to trigger Qodex during merge requests and after changes land on main.

Before you start

You need:
  • A Qodex project with runnable scenarios.
  • A project API key from Settings > Platform > API keys.
  • Permission to add GitLab CI/CD variables.

Add CI/CD variables

In GitLab, open Settings > CI/CD > Variables.
  • QODEX_API_KEY: your qk_... key. Mark it masked and protected when appropriate.
  • QODEX_PROJECT: your Qodex project slug.

Add the job

Save this file as .gitlab-ci.yml.
stages:
  - test

qodex:
  stage: test
  image: alpine:3.20
  timeout: 30m
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      ENV_NAME=staging
      if [ "$CI_COMMIT_REF_NAME" = "main" ]; then ENV_NAME=prod; fi

      RESPONSE=$(curl -fsS -X POST \
        "https://agents.qodex.ai/api/projects/${QODEX_PROJECT}/webhooks/trigger" \
        -H "Authorization: Bearer ${QODEX_API_KEY}" \
        -H "Content-Type: application/json" \
        -d "{\"environment\":\"${ENV_NAME}\",\"tags\":[\"smoke\"]}")

      RUN_ID=$(echo "$RESPONSE" | jq -r .testRunId)
      if [ -z "$RUN_ID" ] || [ "$RUN_ID" = "null" ]; then
        echo "Failed to start run. Response: $RESPONSE"
        exit 1
      fi

      DEADLINE=$(( $(date +%s) + 1500 ))
      while [ "$(date +%s)" -lt "$DEADLINE" ]; do
        STATUS=$(curl -fsS \
          "https://agents.qodex.ai/api/projects/${QODEX_PROJECT}/test-runs/${RUN_ID}" \
          -H "Authorization: Bearer ${QODEX_API_KEY}" | jq -r .status)
        echo "status=$STATUS"
        case "$STATUS" in
          completed) exit 0 ;;
          failed) exit 1 ;;
          *) sleep 10 ;;
        esac
      done
      echo "Timed out waiting for Qodex run ${RUN_ID}"
      exit 1
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

Gate merge requests

Enable Pipelines must succeed in GitLab merge request settings. The Qodex job then becomes part of the merge gate.

Common adjustments

  • Use tags:["critical"] for a faster post-deploy check.
  • Remove tags to run the full active suite.
  • Add a separate deploy-time job that runs only after the deploy stage.

CircleCI recipe

Generic shell recipe