CTRF: Common Test Report Format Setup Guide

CTRF (Common Test Report Format) is a universal JSON schema for test results. It provides a standardized way to report test outcomes across any testing tool, framework, or language.

Why Use CTRF?

• Universal: Works with any test framework that has a CTRF reporter
• Consistent: Same format across Jest, Playwright, pytest, RSpec, and more
• Rich data: Includes timing, retries, flaky test detection, and metadata
• Open standard: Community-driven, MIT-licensed specification at ctrf.io

Supported Frameworks

CTRF has reporters for most popular test frameworks:

Framework	Package	Language
Playwright	playwright-ctrf-json-reporter	JavaScript/TypeScript
Jest	jest-ctrf-json-reporter	JavaScript/TypeScript
Vitest	vitest-ctrf-json-reporter	JavaScript/TypeScript
Cypress	cypress-ctrf-json-reporter	JavaScript/TypeScript
Mocha	mocha-ctrf-json-reporter	JavaScript/TypeScript
pytest	pytest-ctrf	Python
Go	ctrf-go-json-reporter	Go
JUnit XML	junit-to-ctrf	Any (CLI converter, not a reporter)

See the full list at ctrf.io.

Installation

Playwright

npm install playwright-ctrf-json-reporter --save-dev

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['playwright-ctrf-json-reporter', { outputDir: '.', outputFile: 'ctrf-report.json' }],
    ['list'], // Also show in console
  ],
});

The reporter writes to <outputDir>/<outputFile>. outputDir defaults to ctrf, so setting it to . puts the report at ./ctrf-report.json, which is the path the upload examples below use.

Jest

npm install jest-ctrf-json-reporter --save-dev

jest.config.js

module.exports = {
  reporters: [
    'default',
    ['jest-ctrf-json-reporter', { outputDir: '.', outputFile: 'ctrf-report.json' }],
  ],
};

Vitest

npm install vitest-ctrf-json-reporter --save-dev

vitest.config.ts

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: [
      'default',
      ['vitest-ctrf-json-reporter', { outputDir: '.', outputFile: 'ctrf-report.json' }],
    ],
  },
});

pytest

pip install pytest-ctrf

pytest --ctrf ctrf-report.json

Troubleshooting

ENOENT: no such file or directory writing the CTRF report

ENOENT: no such file or directory, open 'ctrf/coverage/report.json'

The cause is putting a directory path inside outputFile. The JS reporters take two separate options and write to path.join(outputDir, outputFile). They create outputDir with mkdirSync(outputDir, { recursive: true }), but nothing creates the directory segments you embed in outputFile, so the write lands in a directory that never got made.

jest.config.js

// ❌ Wrong: resolves to ctrf/coverage/report.json (default outputDir is `ctrf`),
//    and the coverage/ segment is never created
['jest-ctrf-json-reporter', { outputFile: 'coverage/report.json' }]

// ✅ Correct: outputDir is created recursively, outputFile is just the filename
['jest-ctrf-json-reporter', { outputDir: 'ctrf/coverage', outputFile: 'report.json' }]

playwright-ctrf-json-reporter and cypress-ctrf-json-reporter behave identically, including the ctrf default for outputDir. vitest-ctrf-json-reporter uses the same option shape with different defaults (vitest-ctrf/report.json). If a reporter’s README doesn’t document outputDir, add a mkdir -p step before the test run and keep outputFile a plain filename.

The same rule explains the other half of this error: with no options at all, the report is written to ctrf/ctrf-report.json, not ./ctrf-report.json. Point your upload step at the real path or set outputDir: '.'.

tests array missing or empty

If the report is generated but Gaffer reports zero tests parsed, the reporter probably wasn’t invoked.

• Default reporter overridden. The reporters array in jest.config.js (or the equivalent in your framework) replaces the default. Make sure both 'default' and the CTRF reporter are listed, otherwise tests run but no CTRF file is written.
• Tests crashed before reporting. If the suite throws during setup (a beforeAll that fails to connect to a service, for example), some reporters bail before writing the file. Check the CI logs for the underlying error before assuming the reporter is broken.

Uploading CTRF Reports

Once you have a CTRF JSON file, upload it to Gaffer:

curl -X POST https://app.gaffer.sh/api/upload \
  -H "X-API-Key: YOUR_PROJECT_TOKEN" \
  -F "files=@ctrf-report.json" \
  -F 'tags={"commitSha":"abc123","branch":"main"}'

GitHub Actions

- name: Run tests
  run: npm test

- name: Upload CTRF report to Gaffer
  if: always()
  uses: gaffer-sh/gaffer-uploader@v2
  with:
    gaffer_upload_token: ${{ secrets.GAFFER_PROJECT_TOKEN }}
    report_path: ./ctrf-report.json
    commit_sha: ${{ github.sha }}
    branch: ${{ github.ref_name }}

GitLab CI

test:
  script:
    - npm ci
    - npm test
  after_script:
    - |
      curl -X POST https://app.gaffer.sh/api/upload \
        -H "X-API-Key: $GAFFER_PROJECT_TOKEN" \
        -F "files=@ctrf-report.json" \
        -F 'tags={"commitSha":"'"$CI_COMMIT_SHA"'","branch":"'"$CI_COMMIT_REF_NAME"'"}'

CTRF Report Structure

A CTRF report contains standardized test result data:

{
  "results": {
    "tool": {
      "name": "playwright"
    },
    "summary": {
      "tests": 42,
      "passed": 40,
      "failed": 1,
      "pending": 0,
      "skipped": 1,
      "other": 0,
      "start": 1703520000000,
      "stop": 1703520060000
    },
    "tests": [
      {
        "name": "should login successfully",
        "status": "passed",
        "duration": 1234,
        "retries": 0,
        "flaky": false
      },
      {
        "name": "should display dashboard",
        "status": "failed",
        "duration": 5678,
        "message": "Element not found: #dashboard"
      }
    ]
  }
}

Benefits with Gaffer

When you upload CTRF reports to Gaffer, you get:

• Structured analytics: Pass rates, duration trends, flaky test detection
• Cross-framework comparison: Compare results from different test suites
• Failure patterns: See which tests fail most frequently
• Historical tracking: See how tests perform over time

Tip

CTRF is particularly useful when you have multiple test frameworks in your project. Instead of learning different report formats, use CTRF everywhere for consistent analytics.

Next Steps

CI Provider Guides:

• GitHub Actions - Use the official Gaffer Action
• GitLab CI - GitLab pipeline integration
• CircleCI - CircleCI workflow integration
• Jenkins - Jenkins pipeline integration
• Bitbucket Pipelines - Bitbucket integration
• Azure DevOps - Azure Pipelines integration

Reference:

• Upload API - Full API documentation
• cURL Guide - Manual uploads and debugging

Get Started

Gaffer’s free tier includes 500 MB of storage with 7-day retention. Upload your first CTRF report in under 5 minutes.

Start Uploading CTRF Reports - Free