Find the best local LLM for your hardware, ranked by benchmarks

v0.5.13 Feature

This release adds 2 notable features for engineering teams evaluating rollout.

Published 1mo LLM Frameworks

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

ai apple-silicon benchmarks cli gguf gpu

+7 more

huggingface inference llm local-llm ollama python vram

Summary

AI summary

Search terms like "7B" match model parameter size accurately.

Full changelog

HF_ENDPOINT support for Hugging Face model metadata fetches, so users behind a mirror can point whichllm at a compatible Hub endpoint. (#128, #131)
Manual detected-GPU overrides for usable VRAM and bandwidth, useful for iGPU and unified-memory systems where automatic detection is too conservative. (#132, #133)
README guidance for safer first-run flags when users want full-GPU, usable-speed recommendations with extra VRAM headroom.

Search terms such as 7B, 0.5B, and 500M now match model parameter size instead of plain substrings, so qwen 7b no longer returns 1.7B or 30B-A3B by accident. (#107, #126)
GGUF sizing now treats FP16 and ternary TQ1_0 / TQ2_0 quant types correctly, avoiding underestimates for those files. (#125)