Glq

v0.5.2 Feature

This release adds 2 notable features for engineering teams evaluating rollout.

Published 1mo Model Serving & MLOps

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

inference llm model-compression pytorch quantization

Summary

AI summary

Fixed Docker image installation order to resolve CUDA torch ImportError.

Type	Severity	Summary	CVE
Feature	Low	Adds VRAM usage and token throughput metrics to checkpoints table Adds VRAM usage and token throughput metrics to checkpoints table Source: granite4.1:30b@2026-06-05-audit Confidence: low	—
Feature	Low	Introduces benchmarks/bench_tps_vllm.py decode harness for performance measurements Introduces benchmarks/bench_tps_vllm.py decode harness for performance measurements Source: granite4.1:30b@2026-06-05-audit Confidence: low	—
Bugfix	Medium	Fixes Docker image runtime ImportError for missing libcudart.so.13 Fixes Docker image runtime ImportError for missing libcudart.so.13 Source: llm_adapter@2026-06-05 Confidence: high	—

Full changelog

Patch release. The glq library is unchanged from v0.5.1 — this bump fixes the Docker image and adds measured perf docs.

Docker image (ghcr.io/cnygaard/glq-env): install vLLM first so it resolves its own matching-CUDA torch (no torch pin / TORCH_INDEX_URL); base stays CUDA 12.8 to mirror the verified L40S venv. This fixes the broken :0.5.0/:0.5.1 images that failed at runtime with ImportError: libcudart.so.13 under --gpus all. The pip package was unaffected on all versions.

Checkpoints table now lists, per model: VRAM (resident weight footprint) and Tok/s (b1 / b32) — single-stream and 32-way batched decode — measured on a g6e.xlarge (NVIDIA L40S) via vLLM 0.20.2. E.g. SmolLM3-3B: 2.4 GiB, 35 / 654 tok/s.
New benchmarks/bench_tps_vllm.py two-point decode harness backing those numbers.
README notes the image-vLLM fix and points vllm serve at :latest.

pip install glq==0.5.2

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track Glq

Get notified when new releases ship.

About Glq