Skip to content

Glq

v0.7.3 Bugfix

This release fixes issues for SREs watching stability and regressions.

✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

inference llm model-compression pytorch quantization

Summary

AI summary

Fixes trellis checkpoints of gemma-4 E-series models to emit exact‑zero columns for KV‑shared layers, preventing init aborts.

Full changelog

Patch release. One fix since v0.7.2:

Trellis checkpoints of gemma-4 E-series models now serve on vLLM. vLLM constructs the full q/k/v fused linear for every layer, but KV-shared layers (num_kv_shared_layers — e.g. 20 of 35 on E2B) carry no k/v weights in any checkpoint; those shards stayed at their sentinel init and the trellis apply aborted at engine init with trellis_packed must be 2-D. Never-loaded shards now emit exact-zero columns — the shell/e8p paths' de-facto semantics (attention discards those columns) — in both the shard-batched (S4b) and per-shard fallback branches.

  • New GPU unit test reproduces the pre-fix crash and pins both branches (loaded shard bit-identical, unloaded columns exact zeros)
  • Validated end-to-end: gemma-4-E2B-it-trellis-3inst-4bpw ran a full AIME-2026 avg@8 eval (240 thinking generations) on vLLM 0.25.1 under FULL cudagraph
  • The HF eager path was never affected; no kernel or checkpoint-format changes

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track Glq

Get notified when new releases ship.

Sign up free

Related context

Related tools

Beta — feedback welcome: [email protected]