Tokentoll

v0.7.0 Feature

This release adds 3 notable features for engineering teams evaluating rollout.

Published 6d Model Serving & MLOps

View tool

✓ No known CVEs patched

Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

anthropic cost-optimization devtools github-action llm mlops

+3 more

openai python static-analysis

Affected surfaces

auth rbac

Summary

AI summary

Updates Highlights, Example PR comment, and New documentation across a mixed release.

Changes in this release

Type	Severity	Summary	CVE
Feature
Feature	Medium	Adds policy block in .tokentoll.yml with budgets and rules. Adds policy block in .tokentoll.yml with budgets and rules. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Medium	Introduces PASS/WARN/FAIL verdict and blocking-findings list in PR comments. Introduces PASS/WARN/FAIL verdict and blocking-findings list in PR comments. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Medium	Adds `fail_on_policy_violation` option to exit non‑zero on FAIL. Adds `fail_on_policy_violation` option to exit non‑zero on FAIL. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Medium	Adds sensible default excludes for tests, examples, docs, etc. Adds sensible default excludes for tests, examples, docs, etc. Source: llm_adapter@2026-05-30 Confidence: low	—
Feature	Low	Adds new documentation files: policy.md, github-action.md, security.md. Adds new documentation files: policy.md, github-action.md, security.md. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Low	Updates upgrade instructions for [email protected] with fail‑on‑policy‑violation flag. Updates upgrade instructions for [email protected] with fail‑on‑policy‑violation flag. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Low	Documents compatibility: policy block is opt‑in; existing configs unchanged; fail‑on‑policy‑violation defaults to false. Documents compatibility: policy block is opt‑in; existing configs unchanged; fail‑on‑policy‑violation defaults to false. Source: llm_adapter@2026-05-30 Confidence: high	—
Feature	Low	Provides sensible default excludes: tests/, examples/, docs/, cookbook/, benchmarks/, evals/, scripts/, notebooks/ (and singular forms). Provides sensible default excludes: tests/, examples/, docs/, cookbook/, benchmarks/, evals/, scripts/, notebooks/ (and singular forms). Source: granite4.1:30b@2026-05-30-audit Confidence: low	—
Refactor	Low	Removes noisy whole‑repo baseline scan on first‑install PRs; posts short install‑confirmation comment instead. Removes noisy whole‑repo baseline scan on first‑install PRs; posts short install‑confirmation comment instead. Source: llm_adapter@2026-05-30 Confidence: high	—

Full changelog

tokentoll graduates from a cost reporter to a CI gate.

Highlights

Policy verdict: .tokentoll.yml now has a policy block with budgets and rules. Every PR gets a PASS / WARN / FAIL verdict and a blocking-findings list at the top of the comment.
Fail on violation: set policies.fail_on_policy_violation: true (or pass --fail-on-policy-violation on the CLI, or fail-on-policy-violation: true on the GitHub Action). The workflow exits non-zero on FAIL after the PR comment has been posted.
Sensible default excludes: tests/, examples/, docs/, cookbook/, benchmarks/, evals/, scripts/, notebooks/ (and singular forms) are excluded by default. Opt out with use_default_excludes: false.
Quieter first install: the noisy whole-repo baseline scan on first-install PRs is gone. tokentoll now posts a short install-confirmation comment instead.

Policy rules

| Rule | Trigger |
|------|---------|
| budgets.max_monthly_delta_usd | total monthly delta exceeds the threshold |
| budgets.max_callsite_monthly_usd | any new or changed call site exceeds the threshold |
| budgets.max_relative_increase | per-call cost grows by more than this multiplier |
| policies.block_unknown_models | new or modified call site uses an unpriced model |
| policies.fail_on_policy_violation | exit 1 on FAIL (CI gate behavior) |

Full reference in docs/policy.md.

Example PR comment

## tokentoll verdict: FAIL

**Blocking findings (2):**

- src/agent.py:42 - per-call cost grew 15.0x (threshold 5x)
- total monthly delta +$812.00 exceeds budget $250.00

> Required action: revert the regression, raise the threshold in .tokentoll.yml, or add an exemption.

New documentation

docs/policy.md: full policy schema and rule semantics
docs/github-action.md: workflow recipes, inputs/outputs, fork PR risk, SHA pinning
docs/security.md: trust model and permission scope

Upgrade

- uses: Jwrede/[email protected]
  with:
    fail-on-policy-violation: true

pip install --upgrade tokentoll

Compatibility

The new policy block is opt-in. Existing .tokentoll.yml files continue to work unchanged; an empty policy produces PASS with no verdict banner.
The fail-on-policy-violation GitHub Action input defaults to false, so existing installations remain in report-only mode until you flip it.
skip_dynamic_models still defaults to false. Dynamic call sites are priced against per-SDK defaults as before.

123 tests pass. Lint and format clean.

View diff on GitHub

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Share on X Share on Bluesky

Track Tokentoll

Get notified when new releases ship.

About Tokentoll

All releases →