Skip to main content

​
Run tests in CircleCI

Use a CircleCI job to run Qodex against staging for branches and production for main.

​
Before you start

You need:
  • A Qodex project with runnable scenarios.
  • A project API key from Settings > Platform > API keys.
  • A CircleCI project connected to your repository.

​
Add environment variables

In CircleCI, open Project Settings > Environment Variables.
  • QODEX_API_KEY: your qk_... key.
  • QODEX_PROJECT: your Qodex project slug.
Use a CircleCI context if multiple repositories should share the same Qodex secret.

​
Add the config

Save this file at .circleci/config.yml. 
version: 2.1

jobs:
  qodex:
    docker:
      - image: cimg/base:current
    parameters:
      env_name:
        type: string
        default: staging
      tags:
        type: string
        default: smoke
    steps:
      - run:
          name: Trigger Qodex run
          command: |
            sudo apt-get update -qq && sudo apt-get install -y jq curl
            BODY="{\"environment\":\"<< parameters.env_name >>\",\"tags\":[\"<< parameters.tags >>\"]}"
            RESPONSE=$(curl -fsS -X POST \
              "https://agents.qodex.ai/api/projects/${QODEX_PROJECT}/webhooks/trigger" \
              -H "Authorization: Bearer ${QODEX_API_KEY}" \
              -H "Content-Type: application/json" \
              -d "$BODY")
            RUN_ID=$(echo "$RESPONSE" | jq -r .testRunId)
            if [ -z "$RUN_ID" ] || [ "$RUN_ID" = "null" ]; then
              echo "Failed to start run. Response: $RESPONSE"
              exit 1
            fi
            echo "$RUN_ID" > /tmp/qodex_run_id
      - run:
          name: Wait for Qodex
          command: |
            RUN_ID=$(cat /tmp/qodex_run_id)
            DEADLINE=$(( $(date +%s) + 1500 ))
            while [ "$(date +%s)" -lt "$DEADLINE" ]; do
              STATUS=$(curl -fsS \
                "https://agents.qodex.ai/api/projects/${QODEX_PROJECT}/test-runs/${RUN_ID}" \
                -H "Authorization: Bearer ${QODEX_API_KEY}" | jq -r .status)
              echo "status=$STATUS"
              case "$STATUS" in
                completed) exit 0 ;;
                failed) exit 1 ;;
                *) sleep 10 ;;
              esac
            done
            echo "Timed out waiting for Qodex run ${RUN_ID}"
            exit 1

workflows:
  pr:
    jobs:
      - qodex:
          name: qodex-staging-smoke
          env_name: staging
          tags: smoke
          filters:
            branches:
              ignore: main
  prod:
    jobs:
      - qodex:
          name: qodex-prod-smoke
          env_name: prod
          tags: smoke
          filters:
            branches:
              only: main

​
Gate the workflow

Make the Qodex job a required job before deploy steps. If your repository mirrors CircleCI checks back to GitHub, add the CircleCI check to branch protection.

Buildkite recipe

Generic shell recipe