ludwig

v0.17.7 Feature

This release adds 1 notable feature for engineering teams evaluating rollout.

Published 22d LLM Frameworks

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

computer-vision data-centric data-science deep machine-learning deeplearning

+11 more

fine-tuning learning llama llama2 llm llm-training machinelearning mistral natural-language natural-language-processing pytorch

Summary

AI summary

Fixed row‑ordering mismatch and Arrow reduction errors in distributed Ray tests.

Full changelog

Fix TypeError: ArrowExtensionArray bool[pyarrow] does not support reduction 'all' in distributed Ray tests caused by Ray 2.56.0 returning Arrow-backed DataFrames from batch callbacks. Fixed by using .to_numpy() instead of .values when comparing preprocessed column values.
Fix row-ordering mismatch between local and Ray-preprocessed DataFrames in tests. Ray 2.56.0 Arrow-backed list columns (vector, sequence, text indices) produced pyarrow scalars and Python lists whose repr() differed from the tobytes() hash of local numpy arrays, causing wrong row pairings. Fixed by normalizing all array-like values through repr(tolist()) in the sort key so both backing stores produce identical hashes.
Add on_preprocess_progress callback with partition-level tracking (released in 0.17.6); add regression test covering features that call add_feature_data without invoking map_partitions.