Find the best local LLM for your hardware, ranked by benchmarks

Hardening release: every Round 3 fix now has a regression test verified
to fail when reverted, the CI lint pipeline is green again (it was red
for the entire 0.5.1 release), and two correctness bugs found by
stress-testing previously unexercised axes are fixed.

Fixed

--profile vision generation inversion

Text leaderboards don't score VLMs, so the only model with a direct
benchmark hit was a two-generations-old Qwen2-VL-7B, which outranked
the current Qwen3-VL-32B even on an 80 GB H100. A curated
multimodal capability source (MMMU-Pro / MMBench, 2026-05) now scores
the Qwen3-VL / Qwen2.5-VL / Qwen2-VL / Llama-Vision / Phi-vision /
Gemma-3 / Pixtral / InternVL3 lines. Qwen3-VL-32B now leads vision at
73-76; the legacy 7B correctly drops to the low 30s.

Apple Silicon partial-offload speed (~3x under-estimate)

The flat 0.45x partial-offload penalty modelled a discrete GPU
spilling to CPU RAM across PCIe. Apple Silicon shares one unified-memory
pool, so spilled weights stay at full bandwidth. DeepSeek-R1-class
models on M2/M3 Ultra reported ~1.7 t/s when real-world is 4-15; now
0.85x for unified memory, 0.45x kept for discrete GPUs.

CI lint was red for all of 0.5.1

Qwen/Qwen3-Coder-30B-A3B-Instruct was a duplicate key in the
LiveBench fallback (silently scored 62 instead of 58) and 12 files were
unformatted — both broke the Lint job. Fixed; Lint + Tests are now
green on this release commit in actual GitHub CI.

Added

• Round 3 regression suite (tests/test_r3_regressions.py, 20 tests).
  Every test was verified to go red when its fix is reverted — they
  pin real bugs, not the current implementation.
• Benchmark snapshot date shown under every ranking, so a stale
  recommendation is self-evident instead of silently trusted.

CI

• GitHub Actions runners updated to Node 24 (checkout@v5,
  setup-python@v6); Node 20 actions are deprecated from 2026-06.

Full changelog: CHANGELOG.md