Skip to content

Bencher

v0.6.7 Breaking

This release includes 1 breaking change for platform teams planning a safe upgrade.

Published 1mo Pipelines
✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

benchmark benchmarking ci-cd code-quality continuous-benchmarking performance

Affected surfaces

auth breaking_upgrade

ReleasePort's take

Moderate signal
editorial:auto 1mo

Release v0.6.7 adds user‑scoped API keys for fine‑grained authentication and deprecates legacy User API tokens, which will soon return 403 on creation.

Why it matters: User API tokens are deprecated; new token creation returns 403 while existing tokens stay functional until phased out.

Summary

AI summary

Add user-scoped API keys with limited permissions and deprecate legacy API tokens.

Changes in this release

Feature Low

Adds user-scoped API keys (`bencher_user_...`) authenticating as owning user across all JWT‑reachable endpoints except key management.

Adds user-scoped API keys (`bencher_user_...`) authenticating as owning user across all JWT‑reachable endpoints except key management.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Adds `bencher user key {create,list,view,update,revoke}` CLI subcommands and an "API Keys" page in the Console for managing user API keys.

Adds `bencher user key {create,list,view,update,revoke}` CLI subcommands and an "API Keys" page in the Console for managing user API keys.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Allows `--key`/`BENCHER_API_KEY` to accept both project‑scoped and user‑scoped keys; `--project` requirement applies only to project‑scoped keys.

Allows `--key`/`BENCHER_API_KEY` to accept both project‑scoped and user‑scoped keys; `--project` requirement applies only to project‑scoped keys.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Replaces "Tokens" entry in Console account dropdown with "Keys" and hides the "API Tokens" page for accounts created after 15 June 2026.

Replaces "Tokens" entry in Console account dropdown with "Keys" and hides the "API Tokens" page for accounts created after 15 June 2026.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Ensures `bencher run --github-actions` always creates a Bencher Report GitHub Check, failing on active Alerts, and treats creation failures as warnings.

Ensures `bencher run --github-actions` always creates a Bencher Report GitHub Check, failing on active Alerts, and treats creation failures as warnings.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Creates GitHub Check on pull request head commit (`pull_request.head.sha` or `workflow_run.head_sha`) instead of `GITHUB_SHA`.

Creates GitHub Check on pull request head commit (`pull_request.head.sha` or `workflow_run.head_sha`) instead of `GITHUB_SHA`.

Source: llm_adapter@2026-06-15

Confidence: high

Feature Low

Appends `--ci-id` ID to GitHub Check name (e.g., `Bencher Report (<ID>)`) for distinct, stable check names per invocation.

Appends `--ci-id` ID to GitHub Check name (e.g., `Bencher Report (<ID>)`) for distinct, stable check names per invocation.

Source: llm_adapter@2026-06-15

Confidence: high

Deprecation High

Deprecates User API tokens in favor of user API keys; token creation endpoint now returns 403 Forbidden while existing tokens remain functional.

Deprecates User API tokens in favor of user API keys; token creation endpoint now returns 403 Forbidden while existing tokens remain functional.

Source: llm_adapter@2026-06-15

Confidence: high

Bugfix Medium

Fixes OCI registry token endpoint for Docker 29+ containerd image store handling multiple scope parameters and OAuth2 POST exchange.

Fixes OCI registry token endpoint for Docker 29+ containerd image store handling multiple scope parameters and OAuth2 POST exchange.

Source: llm_adapter@2026-06-15

Confidence: high

Full changelog
  • Fix the OCI registry token endpoint for the Docker 29+ containerd image store, which sends multiple scope query parameters and an OAuth2 POST token exchange (Thank you @travishathaway)
  • Add user-scoped API keys (bencher_user_...) that authenticate as the owning user across every endpoint a JWT can reach, including docker login; the one exception is key management: a user API key cannot create more keys and can only see, update, or revoke itself, so a leaked key cannot mint credentials that outlive its own revocation, tamper with the user's other keys, or enumerate them
  • Add bencher user key {create,list,view,update,revoke} CLI subcommands and an "API Keys" page in the Console for managing user API keys
  • --key/BENCHER_API_KEY now accepts either a project-scoped key (bencher_run_*) or a user-scoped key (bencher_user_*); the --project requirement applies only to project-scoped keys
  • DEPRECATED User API tokens are deprecated in favor of user API keys: POST /v0/users/{user}/tokens (bencher token create) now always fails with a 403 Forbidden error; existing API tokens continue to work and can still be listed, viewed, renamed, and revoked
  • Replace the Tokens entry in the Console account dropdown with Keys, and hide the "API Tokens" page entirely for accounts created after 15 June 2026
  • bencher run --github-actions now always creates a Bencher Report GitHub Check (failing on an active Alert), so it can be used as a required status check in branch protection; the check is created on a best effort basis, so check creation failures are now warnings instead of errors (Thank you @OmarTawfik)
  • Create the GitHub Check on the pull request head commit (pull_request.head.sha or workflow_run.head_sha) instead of GITHUB_SHA, so the check appears on the pull request
  • Append the --ci-id ID to the GitHub Check name (ie Bencher Report (<ID>)) so multiple bencher run invocations on the same commit get distinct, stable check names

Breaking Changes

  • DEPRECATED: `POST /v0/users/{user}/tokens` (`bencher token create`) now always returns 403 Forbidden; existing tokens remain functional.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track Bencher

Get notified when new releases ship.

Sign up free

About Bencher

Suite of continuous benchmarking tools designed to catch performance regressions in CI.

All releases →

Related context

Earlier breaking changes

  • v0.6.9 Omit results and alerts from reports list endpoint by default.

Beta — feedback welcome: [email protected]