Rubric

v0.1.1 Breaking

This release includes breaking changes for platform teams planning a safe upgrade.

Published 4mo LLM Frameworks

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

agents ai anthropic benchmarking evals evaluation

+7 more

langchain llm llm-evaluation machine-learning openai pytest testing

Summary

AI summary

Renamed TestResult.score to TestResult.overall_score and made CLI output flags work correctly.

Full changelog

Patch release fixing 5 bugs found during external testing, plus corrected PyPI project links.

TestResult.score → renamed to TestResult.overall_score (was causing AttributeError for users following the README Quick Start)
- CLI --output-html / --output-json flags now correctly write report files (env vars were set but never read by evaluate())
- raise_on_failure=True now calls sys.exit(1) instead of raising AssertionError — CI/CD pipelines expecting a non-zero exit code now work correctly
- pytest_plugin.py hard import pytest replaced with a guarded import — no longer crashes when pytest is not installed
- Reason text in live console output no longer truncated at 80 characters
- Fixed all PyPI project links (Homepage, Repository, Bug Tracker, Changelog, Docs) to point to correct GitHub URL

All v0.1.0 code is fully compatible.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track Rubric

Get notified when new releases ship.

About Rubric