CTRF Guide: playwright-ctrf-json-reporter, jest-ctrf, vitest-ctrf & More

CTRF (Common Test Report Format) is a universal JSON schema for test results. It provides a standardized way to report test outcomes across any testing tool, framework, or language.

Why Use CTRF?

• Universal: Works with any test framework that has a CTRF reporter
• Consistent: Same format across Jest, Playwright, pytest, RSpec, and more
• Rich data: Includes timing, retries, flaky test detection, and metadata
• Open standard: Community-driven, MIT-licensed specification at ctrf.io

Supported Frameworks

CTRF has reporters for most popular test frameworks:

Framework	Package	Language
Playwright	playwright-ctrf-json-reporter	JavaScript/TypeScript
Jest	jest-ctrf-json-reporter	JavaScript/TypeScript
Vitest	vitest-ctrf-json-reporter	JavaScript/TypeScript
Cypress	cypress-ctrf-json-reporter	JavaScript/TypeScript
Mocha	mocha-ctrf-json-reporter	JavaScript/TypeScript
pytest	pytest-ctrf	Python
Go	ctrf-go-json-reporter	Go
JUnit	junit-json-reporter-ctrf	Java

See the full list at ctrf.io.

Installation

Playwright

npm install playwright-ctrf-json-reporter --save-dev

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['playwright-ctrf-json-reporter', { outputFile: 'ctrf-report.json' }],
    ['list'], // Also show in console
  ],
});

Jest

npm install jest-ctrf-json-reporter --save-dev

jest.config.js

module.exports = {
  reporters: [
    'default',
    ['jest-ctrf-json-reporter', { outputFile: 'ctrf-report.json' }],
  ],
};

Vitest

npm install vitest-ctrf-json-reporter --save-dev

vitest.config.ts

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: ['default', 'vitest-ctrf-json-reporter'],
  },
});

pytest

pip install pytest-ctrf

pytest --ctrf ctrf-report.json

Uploading CTRF Reports

Once you have a CTRF JSON file, upload it to Gaffer:

curl -X POST https://app.gaffer.sh/api/upload \
  -H "X-API-Key: YOUR_UPLOAD_TOKEN" \
  -F "files=@ctrf-report.json" \
  -F 'tags={"commitSha":"abc123","branch":"main"}'

GitHub Actions

- name: Run tests
  run: npm test

- name: Upload CTRF report to Gaffer
  if: always()
  uses: gaffer-sh/gaffer-uploader@v1
  with:
    gaffer_api_key: ${{ secrets.GAFFER_UPLOAD_TOKEN }}
    report_path: ./ctrf-report.json
    commit_sha: ${{ github.sha }}
    branch: ${{ github.ref_name }}

GitLab CI

test:
  script:
    - npm ci
    - npm test
  after_script:
    - |
      curl -X POST https://app.gaffer.sh/api/upload \
        -H "X-API-Key: $GAFFER_UPLOAD_TOKEN" \
        -F "files=@ctrf-report.json" \
        -F 'tags={"commitSha":"'"$CI_COMMIT_SHA"'","branch":"'"$CI_COMMIT_REF_NAME"'"}'

CTRF Report Structure

A CTRF report contains standardized test result data:

{
  "results": {
    "tool": {
      "name": "playwright"
    },
    "summary": {
      "tests": 42,
      "passed": 40,
      "failed": 1,
      "pending": 0,
      "skipped": 1,
      "other": 0,
      "start": 1703520000000,
      "stop": 1703520060000
    },
    "tests": [
      {
        "name": "should login successfully",
        "status": "passed",
        "duration": 1234,
        "retries": 0,
        "flaky": false
      },
      {
        "name": "should display dashboard",
        "status": "failed",
        "duration": 5678,
        "message": "Element not found: #dashboard"
      }
    ]
  }
}

Benefits with Gaffer

When you upload CTRF reports to Gaffer, you get:

• Structured analytics: Pass rates, duration trends, flaky test detection
• Cross-framework comparison: Compare results from different test suites
• Failure patterns: See which tests fail most frequently
• Historical tracking: See how tests perform over time

Tip

CTRF is particularly useful when you have multiple test frameworks in your project. Instead of learning different report formats, use CTRF everywhere for consistent analytics.

Next Steps

CI Provider Guides:

• GitHub Actions - Use the official Gaffer Action
• GitLab CI - GitLab pipeline integration
• CircleCI - CircleCI workflow integration
• Jenkins - Jenkins pipeline integration
• Bitbucket Pipelines - Bitbucket integration
• Azure DevOps - Azure Pipelines integration

Reference:

• Upload API - Full API documentation
• cURL Guide - Manual uploads and debugging

Get Started

Gaffer’s free tier includes 500 MB of storage with 7-day retention. Upload your first CTRF report in under 5 minutes.

Start Uploading CTRF Reports - Free