OpenTelemetry
Gaffer can export test run and coverage metrics as OpenTelemetry (OTLP) data to your existing observability stack. Track pass rates, failure counts, and coverage trends alongside your service metrics in Datadog, Grafana Cloud, or any OTLP-compatible backend.
Features
Section titled “Features”- Provider presets — built-in support for Datadog and Grafana Cloud, plus a generic OTLP option for any compatible endpoint
- Branch filtering — limit metric export to specific branches using glob patterns
- Test run metrics — pass rate, total/passed/failed/skipped counts per run
- Coverage metrics — line and branch coverage percentages per upload
- Test connectivity — verify your configuration from the dashboard before waiting for real data
- Encrypted API keys — credentials are encrypted at rest with AES-256-GCM
Setup by Provider
Section titled “Setup by Provider”Datadog
Section titled “Datadog”- In Datadog, go to Organization Settings > API Keys and create (or copy) an API key
- In Gaffer, navigate to Settings > OpenTelemetry and click “Add destination”
- Select Datadog as the provider
- Paste your API key and select your Datadog site (e.g.,
datadoghq.com,datadoghq.eu,us3.datadoghq.com,us5.datadoghq.com,ap1.datadoghq.com) - Choose which event types to export (test runs, coverage, or both)
- Click “Create”
- Click “Test” to verify connectivity
- Verify in Datadog: go to Metrics > Explorer and search for
gaffer.tests.pass_rate
Grafana Cloud
Section titled “Grafana Cloud”- In Grafana Cloud, go to your stack’s OpenTelemetry section to get your OTLP gateway credentials (instance ID, region, API token)
- In Gaffer, navigate to Settings > OpenTelemetry and click “Add destination”
- Select Grafana as the provider
- Enter your Grafana Cloud region, instance ID, and API token
- Choose event types and click “Create”
- Click “Test” to verify connectivity
- Verify in Grafana: go to Explore and query for
gaffer_tests_pass_rate
Generic OTLP
Section titled “Generic OTLP”Any backend that accepts OTLP over HTTP can be used as a destination.
- In Gaffer, navigate to Settings > OpenTelemetry and click “Add destination”
- Select Generic OTLP as the provider
- Enter your OTLP HTTP endpoint URL (must be HTTPS)
- Enter an API key or bearer token for authentication
- Choose event types and click “Create”
- Click “Test” to verify connectivity
Configuration Options
Section titled “Configuration Options”Event Types
Section titled “Event Types”Choose which events export metrics:
- Test runs — exports pass rate, total/passed/failed/skipped counts when a test run completes
- Coverage — exports line and branch coverage percentages when a coverage report is uploaded
At least one event type must be enabled.
Branch Filters
Section titled “Branch Filters”Restrict which branches trigger metric export using glob patterns:
main— exact matchrelease/*— matchesrelease/v1.0,release/2024-01, etc.feature-*— matchesfeature-auth,feature-dashboard, etc.
Leave branch filters empty to export metrics for all branches.
Enabled Toggle
Section titled “Enabled Toggle”Each destination can be enabled or disabled independently. Disabled destinations remain configured but do not export metrics.
Available Metrics
Section titled “Available Metrics”Test Run Metrics
Section titled “Test Run Metrics”| Metric | Type | Unit | Description |
|---|---|---|---|
gaffer.tests.pass_rate | Gauge | % | Pass rate of the test run |
gaffer.tests.total | Gauge | test | Total test count |
gaffer.tests.passed | Gauge | test | Passed test count |
gaffer.tests.failed | Gauge | test | Failed test count |
gaffer.tests.skipped | Gauge | test | Skipped test count |
Coverage Metrics
Section titled “Coverage Metrics”| Metric | Type | Unit | Description |
|---|---|---|---|
gaffer.coverage.line_percent | Gauge | % | Line coverage percentage |
gaffer.coverage.branch_percent | Gauge | % | Branch coverage percentage |
Dimensions (Attributes)
Section titled “Dimensions (Attributes)”Metrics include these attributes for filtering and grouping:
| Attribute | Included On | Description |
|---|---|---|
gaffer.project.id | All metrics | Project identifier |
gaffer.project.name | All metrics | Project name |
gaffer.branch | All metrics | Git branch name (when available) |
gaffer.commit_sha | All metrics | Git commit SHA (when available) |
gaffer.report_format | Test run metrics only | Report format (e.g., ctrf) |
Dashboard Templates
Section titled “Dashboard Templates”Get started faster with a pre-built Datadog dashboard. See Datadog Test Metrics Dashboard for a downloadable JSON template and import instructions.
Testing
Section titled “Testing”Each destination has a “Test” button in the dashboard. Clicking it sends a test metric payload to your configured endpoint and reports the HTTP status code. Use this to verify connectivity and authentication before waiting for real test data.
Troubleshooting
Section titled “Troubleshooting”Test button returns an error?
Section titled “Test button returns an error?”- Check your API key — make sure the key is correct and has the necessary permissions
- Verify the site/region — Datadog and Grafana Cloud require the correct region to be selected
- Check endpoint URL — for generic OTLP, ensure the URL is correct and reachable
- Network issues — if your backend is behind a firewall, ensure it accepts connections from Gaffer’s servers
Metrics not appearing in your backend?
Section titled “Metrics not appearing in your backend?”- Check the destination is enabled — disabled destinations do not export metrics
- Verify branch filters — if configured, only matching branches trigger export
- Check event types — make sure the relevant event type (test runs or coverage) is enabled
- Check “Last Sent” — the destinations list shows the last delivery time and any errors
- Allow propagation time — some backends take a few minutes to index new metrics
Security
Section titled “Security”- Encrypted API keys — credentials are encrypted at rest using AES-256-GCM
- HTTPS recommended — Datadog and Grafana Cloud presets use HTTPS endpoints; for generic OTLP, use HTTPS to protect data in transit