Find the best local LLM for your hardware, ranked by benchmarks

v0.5.7 Feature

This release adds 3 notable features for engineering teams evaluating rollout.

Published 15d LLM Frameworks

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

ai apple-silicon benchmarks cli gguf gpu

+7 more

huggingface inference llm local-llm ollama python vram

Summary

AI summary

Fix crashes when running large Transformer models with an offload_folder.

Type	Severity	Summary	CVE
Feature
Feature	Medium	Detect DGX Spark / NVIDIA GB10 as shared-memory NVIDIA GPU when memory.total unavailable. Detect DGX Spark / NVIDIA GB10 as shared-memory NVIDIA GPU when memory.total unavailable. Source: granite4.1:8b-q6_K@2026-05-19 Confidence: high	—
Feature	Medium	Respect XDG_CACHE_HOME for cache paths, ignoring relative values per XDG spec. Respect XDG_CACHE_HOME for cache paths, ignoring relative values per XDG spec. Source: granite4.1:8b-q6_K@2026-05-19 Confidence: high	—
Feature	Medium	Treat Apple Silicon as shared memory in fit detection. Treat Apple Silicon as shared memory in fit detection. Source: granite4.1:8b-q6_K@2026-05-19 Confidence: high	—
Performance	Medium	Inline LiveBench fallback data and speed up benchmark score fetching. Inline LiveBench fallback data and speed up benchmark score fetching. Source: granite4.1:8b-q6_K@2026-05-19 Confidence: high	—
Bugfix	Medium	`whichllm run` no longer crashes for large Transformers models with offload_folder. `whichllm run` no longer crashes for large Transformers models with offload_folder. Source: granite4.1:8b-q6_K@2026-05-19 Confidence: high	—

Full changelog

Detect DGX Spark / NVIDIA GB10 as a shared-memory NVIDIA GPU when NVIDIA reports memory.total as unavailable.
Fix whichllm run crashes for large Transformers models by providing an offload_folder.
Respect XDG_CACHE_HOME for cache paths, while ignoring relative values per the XDG spec.
Treat Apple Silicon as shared memory in fit detection.
Inline LiveBench fallback data and speed up benchmark score fetching.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track Find the best local LLM for your hardware, ranked by benchmarks

Get notified when new releases ship.

About Find the best local LLM for your hardware, ranked by benchmarks