UQLM

These release notes are for minor release v0.2.0.

New Features

1. Progress bars with rich

This feature enables the use of progress bars when generating LLM responses, scoring responses, and tuning ensemble weights. This feature introduced rich and ipywidgets as new dependencies.
By default, progress bars are turned on, but users can turn them off by setting show_progress_bars=False in generate_and_score, score, and tune methods for the scorer classes. Below is a screenshot illustrating the use of rich progress bars with the UQEnsemble.tune method:

2. Ensemble weights printing

After running the UQEnsemble.tune method, ensemble weights are now printed in a pretty table using rich. Ensemble weights are sorted from highest to lowest. See the above screenshot for an example. Users can also display this table with an already tuned ensemble using the UQEnsemble.print_weights method.

3. Support for Python 3.13

As of v0.2.0, uqlm can now be used with Python 3.13. All previous functionality is supported except for bleurt, which is not compatible with Python 3.13.

4. Ensemble saving and loading

UQEnsemble now offers two new methods: save_config and load_config. These methods offer user-friendly saving and loading the ensemble scorer components and weights.

Example use of ensemble saving:

uqe_tuned_config_file = "uqe_config_tuned.json"uqe.save_config(uqe_tuned_config_file)

Example use of ensemble loading:

loaded_ensemble = UQEnsemble.load_config("uqe_config_tuned.json")

These methods make storing a tuned ensemble an easier process for later use.

5. Token-probability-based Semantic Entropy

The SemanticEntropy class now supports token-probability-based estimates of semantic entropy and associated confidence scores. Note that attribute names in the returned object and column names in the associated dataframe have changed from those in v0.1.

Breaking Changes

1. BLEURT Deprecation

This release deprecates BLEURT as a black-box scorer. The following code will now produce errors:

• Use of uqlm.black_box.BLEURTScorer
• Use of "bleurt" in uqlm.scorers.BlackBoxUQ scorers parameter
• Use of "bleurt" in uqlm.scorers.UQEnsemble scorers parameter