Bencher

Pipelines

A continuous benchmarking platform that runs benchmarks on identical bare‑metal hardware locally and in CI to detect performance regressions.

Rust Latest v0.6.6 · 17d ago Security brief →

Features

Run the same bare‑metal benchmarks both locally and in CI for consistent results
Track, query, and graph benchmark data over time via a web console
Automatically detect and fail PRs on performance regressions with customizable alerts

View all 16 releases →

Review required

v0.6.6 Breaking risk 17d

Auth

Rename usage to metrics

Open

v0.6.5 Bug fix 23d

Notable features

Full changelog

Add project scoped API keys
Add build time and file size tracking support for bare metal runs
Fix bencher run hanging on stdin when no command is provided, including for --image remote bare metal runs (Thank you @sophie-h)

v0.6.4 Breaking risk 1mo

Breaking changes

Switched runner key format from hex encoding (79 characters) to alphanumeric encoding (45 characters); existing keys must be rotated.

Full changelog

Allow a minimum sample size of 1 for the Percentage (percentage) Threshold Test (Thank you @teofr)
BREAKING CHANGE Switch runner key format from hex to alphanumeric encoding, reducing key length from 79 to 45 characters; existing runner keys must be rotated

v0.6.3 Breaking risk 1mo

Breaking changes

Rename `runner token` to `runner key` across the API, CLI, and environment variables

Notable features

Add API token revocation with `bencher token revoke` and Revoke button in the Console

Full changelog

Fix sample standard deviation for Z-score, t-test, and Log Normal Thresholds to use Bessel's correction (divide by n − 1) (Thank you @ggirol-rc)
Fix t-test Threshold to use the prediction interval (scale the standard deviation by √(1 + 1/n)) so the boundary tests whether a new Metric is consistent with the historical distribution rather than whether it contains the historical mean (Thank you @ggirol-rc)
Existing Thresholds will produce slightly wider (statistically correct) boundaries; no configuration changes required
Fix Alert JSON to return the Spec for the Report in which the Alert was generated instead of the Testbed's current Spec
BREAKING CHANGE Rename runner token to runner key across the API, CLI, and environment variables
Fix race condition where concurrent branch creation could cause a "branch head is null" 500 error (Thank you @vmelamed)
Self-heal legacy Branches whose head_id is still NULL from the race condition above so subsequent bencher run submissions succeed instead of 500ing (Thank you @vmelamed)
Add API token revocation with bencher token revoke and Revoke button in the Console

v0.6.2 Bug fix 1mo

Fix URL fragment ordering per RFC 3986 and fallback for missing --start-point-hash.

Full changelog

Allow unauthenticated docker push to unclaimed projects
Fix URL fragment placed before query string per RFC 3986 (Thank you @ricardoboss)
Fix --start-point-hash failure on non-existent hash by falling back to the start point branch's latest version (Thank you @teofr)

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.