CircleCI

CircleCI’s own test insights only cover a single project’s build history inside CircleCI. Gaffer adds cross-build history, flaky detection, and a report link that works for anyone, CircleCI account or not.

Prerequisites

• A Gaffer account with a project
• Your project token
• A CircleCI project with a config.yml file

How do I add my Gaffer project token to CircleCI?

Add it as a project environment variable, or a context if you have more than one CircleCI project.

For a single project:

1. Go to your CircleCI project settings
2. Navigate to Environment Variables
3. Click Add Environment Variable
4. Name: GAFFER_PROJECT_TOKEN
5. Value: your Gaffer project token

How do I share one project token across multiple CircleCI projects?

Use a CircleCI context instead of a project-level environment variable. Contexts are defined once at the organization level and attached to jobs in any project, so you set GAFFER_PROJECT_TOKEN once instead of re-entering it per project.

1. Go to Organization Settings → Contexts
2. Create a context, e.g. gaffer
3. Add GAFFER_PROJECT_TOKEN as an environment variable on the context
4. Reference it in each project’s config.yml with context: gaffer

How do I upload a report after a CircleCI job runs?

Add a run step with when: always. Without it, the step is skipped whenever an earlier step in the job fails, which is exactly when you want the report.

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.0
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm ci
      - run:
          name: Run tests
          command: npm test
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@test-results/junit.xml" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'"}'
      - store_test_results:
          path: test-results

workflows:
  test:
    jobs:
      - test

Keeping store_test_results alongside the Gaffer upload is intentional: it feeds CircleCI’s own Insights timing charts for this one project, while Gaffer keeps the report and history independent of CircleCI.

Which CircleCI variables carry commit and branch info?

Variable	Description	Example
$CIRCLE_SHA1	Full commit SHA	abc123def456...
$CIRCLE_BRANCH	Branch name (empty on tag-triggered builds)	main, feature/login
$CIRCLE_TAG	Git tag name (only set on tag-triggered builds)	v1.2.0
$CIRCLE_PR_NUMBER	Pull request number (only available on forked PRs)	42
$CIRCLE_NODE_INDEX	Index of the current container under parallelism	0, 1, 2
$CIRCLE_BUILD_NUM	Build number	123

Framework examples

Playwright

version: 2.1

jobs:
  playwright:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-jammy
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm ci
      - run:
          name: Run Playwright tests
          command: npx playwright test
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@playwright-report/index.html" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'","test_framework":"playwright","test_suite":"e2e"}'
      - store_artifacts:
          path: playwright-report

Jest with JUnit reporter

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.0
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm ci
      - run:
          name: Run Jest tests
          command: npm test -- --reporters=default --reporters=jest-junit
          environment:
            JEST_JUNIT_OUTPUT_DIR: ./test-results
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@test-results/junit.xml" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'","test_framework":"jest"}'
      - store_test_results:
          path: test-results

pytest

version: 2.1

jobs:
  pytest:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: pip install pytest pytest-html
      - run:
          name: Run pytest
          command: pytest --html=report.html --self-contained-html
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@report.html" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'","test_framework":"pytest"}'
      - store_artifacts:
          path: report.html

Using CTRF format

For a standardized format across all your test frameworks, use CTRF:

- run:
    name: Install CTRF reporter
    # Choose the reporter for your framework:
    # npm install --save-dev jest-ctrf-json-reporter
    # npm install --save-dev playwright-ctrf-json-reporter
    # npm install --save-dev vitest-ctrf-json-reporter
    command: npm install --save-dev jest-ctrf-json-reporter
- run:
    name: Run tests with CTRF
    command: npm test -- --reporter=jest-ctrf-json-reporter
- run:
    name: Upload CTRF to Gaffer
    when: always
    command: |
      curl -X POST https://app.gaffer.sh/api/upload \
        -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
        -F "files=@ctrf-report.json" \
        -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'"}'

Does CircleCI’s own test insights replace Gaffer?

No, they cover different scope. store_test_results feeds CircleCI Insights, which summarizes timing and flake rate for a single project’s own build history, visible only inside that CircleCI org. Gaffer keeps a report you can link to someone without CircleCI access, and correlates runs against the commit and branch tags you send rather than CircleCI’s internal job graph. Keep store_test_results for CircleCI’s native charts and add Gaffer for anything that needs to leave CircleCI, like a report link in a customer ticket or a Slack alert.

Is there an official Gaffer CircleCI orb?

No. Gaffer publishes a GitHub Action and a CLI, but not a CircleCI orb. On CircleCI, the two supported paths are calling the upload API directly with curl, as every example on this page does, or running the gaffer CLI as a normal run step. If you want to avoid repeating the same run block across jobs, define it once as a reusable command in CircleCI config 2.1 rather than reaching for a third-party orb:

version: 2.1

commands:
  upload-to-gaffer:
    parameters:
      report_path:
        type: string
      test_framework:
        type: string
        default: ""
    steps:
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@<< parameters.report_path >>" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"${CIRCLE_BRANCH:-$CIRCLE_TAG}"'","test_framework":"<< parameters.test_framework >>"}'

jobs:
  test:
    docker:
      - image: cimg/node:20.0
    steps:
      - checkout
      - run: npm ci
      - run: npm test
      - upload-to-gaffer:
          report_path: test-results/junit.xml
          test_framework: jest

Why is $CIRCLE_BRANCH empty on some builds?

CircleCI only populates $CIRCLE_BRANCH on branch-triggered builds. A build triggered by pushing a git tag sets $CIRCLE_TAG instead and leaves $CIRCLE_BRANCH unset. If your release pipeline runs on tags, an upload that only reads $CIRCLE_BRANCH sends an empty branch tag to Gaffer. Fall back explicitly:

BRANCH="${CIRCLE_BRANCH:-$CIRCLE_TAG}"
curl -X POST https://app.gaffer.sh/api/upload \
  -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
  -F "files=@test-results/junit.xml" \
  -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$BRANCH"'"}'

How do I upload from CircleCI’s parallel test splitting without runs colliding?

Tag each container’s upload with $CIRCLE_NODE_INDEX. Gaffer records each curl call as an independent test run; parallelism fans the same job out across N containers, and without a per-container tag those N uploads land as N indistinguishable runs against the same commit.

jobs:
  test:
    parallelism: 4
    steps:
      - run:
          name: Run tests
          command: |
            circleci tests glob "**/*.spec.js" | circleci tests split | xargs npm test --
      - run:
          name: Upload to Gaffer
          when: always
          command: |
            curl -X POST https://app.gaffer.sh/api/upload \
              -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
              -F "files=@test-results/junit.xml" \
              -F 'tags={"commitSha":"'"$CIRCLE_SHA1"'","branch":"'"$CIRCLE_BRANCH"'","container":"'"$CIRCLE_NODE_INDEX"'"}'

What goes wrong and how do I fix it?

Symptom	Cause	Fix
Upload step never runs	Missing when: always on the upload run step	Add when: always, CircleCI skips steps by default once an earlier step fails
Branch shows as empty in Gaffer	Build triggered by a tag, not a branch push, and $CIRCLE_BRANCH is unset in that case	Fall back with ${CIRCLE_BRANCH:-$CIRCLE_TAG}
401 Unauthorized on forked-PR builds	CircleCI withholds project environment variables from fork-triggered builds by design	Skip the upload on fork builds (see below). Do not turn on “Pass secrets to builds from forked pull requests” to work around this
Same commit shows N nearly-identical runs	parallelism fans a job across containers, and each container uploads independently	Add a container or similar tag using $CIRCLE_NODE_INDEX
401 Unauthorized, variable looks correct	Token missing the gfr_ prefix, or the variable is defined on the wrong project/context	Re-copy the token, confirm the context is attached to the job if you’re using one

Don't pass secrets to fork builds

CircleCI hides project environment variables from fork-triggered builds deliberately. Enabling “Pass secrets to builds from forked pull requests” hands GAFFER_PROJECT_TOKEN to any code in any pull request opened from a fork, and that token can read your project’s test data, upload forged reports, and abort in-flight uploads. On a public repository, treat it as publishing the token.

Guard the upload instead. $CIRCLE_PR_NUMBER is set only on forked pull requests, so skipping on it costs you fork coverage and nothing else:

- run:
    name: Upload to Gaffer
    when: always
    command: |
      if [ -n "$CIRCLE_PR_NUMBER" ]; then
        echo "Fork PR: no project token available, skipping upload"
        exit 0
      fi
      curl -sS -X POST https://app.gaffer.sh/api/upload \
        -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
        -F "files=@test-results/junit.xml"

If you do need reports from fork contributions, have a maintainer re-run the pipeline on a branch in the main repository once the change is reviewed.

Next Steps

• CTRF Guide - Use the universal test format
• Upload API Reference - Full API documentation
• Slack Integration - Get test results in Slack

Other CI Providers: GitHub Actions · GitLab CI · Jenkins · Bitbucket · Azure DevOps