UQLM

v0.2.3 Breaking

This release includes 1 breaking change for platform teams planning a safe upgrade.

Published 10mo AI Agents & Assistants

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

ai-evaluation ai-safety confidence-estimation confidence-score hallucination hallucination-detection

+8 more

hallucination-evaluation hallucination-mitigation llm llm-evaluation llm-hallucination llm-safety uncertainty-estimation uncertainty-quantification

Summary

AI summary

BERTScorer replaces the slow bert_score.score function, adding a configurable response format option and removing inconsistent color formatting.

Full changelog

Replaces use of bert_score.score with bert_score.BERTScorer.score for a ~43x speedup. While the former (old approach) re-checks and re-assigns torch.device with each use of score, the latter (updated approach) assigns torch.device only once during instantiation.
Creates the option for users to specify whether they want only postprocessed responses, only raw responses, or both versions when they specify a postprocessor. This applies to BlackBoxUQ, UQEnsemble, and SemanticEntropy. To do so, users can respectively specify 'postprocessed', 'raw', or 'all' in the 'return_responses' argument in the constructor of these classes. By default, 'all' is specified.
[black] is removed where specified in rich print statements to avoid inconsistent colors in progress bars.

v0.2.2 updates by @dylanbouchard in https://github.com/cvs-health/uqlm/pull/140
use bert_score class rather than function for 43x speedup by @dylanbouchard in https://github.com/cvs-health/uqlm/pull/141
Enable different handling of raw vs postprocessed responses by @dylanbouchard in https://github.com/cvs-health/uqlm/pull/143
Patch release: v0.2.3 by @dylanbouchard in https://github.com/cvs-health/uqlm/pull/144

Full Changelog: https://github.com/cvs-health/uqlm/compare/v0.2.2...v0.2.3

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track UQLM

Get notified when new releases ship.

About UQLM