Glq
Model Serving & MLOpsPost‑training weight quantization for large language models using E8 lattice codebooks, achieving 2–8 bits‑per‑weight with performance comparable to state‑of‑the‑art methods.
Features
- Encodes each 8‑weight group as a 16‑bit index into an E8 lattice codebook
- Uses Randomized Hadamard Transform for decorrelation, enabling near‑optimal Euclidean nearest‑neighbour search
- Provides fused CUDA kernels that matmul directly against compressed indices without full dequantization
- Supports flexible bit‑width quantization (2–8 bpw, including fractional values)
- Includes mixed‑precision allocation flow for per‑layer bpw tuning
Recent releases
View all 24 releases →Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Install & Platforms
Install via
pip
Platforms
linux
macos
windows
arm64
