Highlights

GLM-5.2 NVFP4, tuned for production: We took time this cycle to tune GLM-5.2 NVFP4 on Blackwell for optimized production serving. It now runs at 500+ tok/s/user on 8x B300, 450 on 4x GB300 (bs=1). Run GLM-5.2 with our cookbook.

• Spec V2 by default: zero-overhead scheduling via CUDA-graphable DSA draft-extend, dropped D2H/H2D syncs, fused metadata ops. +11% end-to-end TPS (#29413, #29343, #29166, #29077).
• IndexShare MTP: reuses the indexer top-k across draft steps, up to 1.9x lower draft-step cost at long context (#29959, #29787, #29654).
• TopK V2: fuses top-k selection with the page-table transform, runtime k up to 2048 (#30274, #26788).
• Indexer prologue fusion: 12 kernels to 4, ~8% faster decode at bs=1 (#27705).
• GEMM: shape-specialized JIT router GEMM + CuteDSL BF16 GEMM for Blackwell (#21531, #30117).
• FlashInfer autotune: now covers draft-model graphs (#29595).

New Model Support: Hunyuan 3 (Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, Baidu Unlimited-OCR, JoyEcho multi-shot A/V, plus Qwen3.6 NVFP4 support.

Native web search (Exa): Built-in web_search support backed by Exa (#29342).

Breakable CUDA Graph on by default: Breakable CUDA Graph is now the default capture path, reducing per-step kernel-launch overhead (#29458); full CUDA Graph support for the prefill phase lands as experimental (#27988).

Linear-attention kernels (KDA / GDN): New FlashKDA prefill backend for safe-gate KDA linear attention (#29472), plus ReplaySSM buffered output-only decode for linear attention (#28451).

FlashInfer A2A for routed MoE: Adds FlashInfer all-to-all with the flashinfer_trtllm_routed MoE runner (#22394).

DeepSeek-V4 Optimization:

• Optimizes C128 state-pool allocation using the request state pool (#28612).
• FlashMLA sparse prefill is now enabled by default for DeepSeek-V4, reaching >10% throghput gain on long context. (#29775).
• Non paged indexer support for long context prefill, with >5% e2e throughput gain.(#29619).

Decode Context Parallelism: decode context parallelism lands for MLA models, including DeepSeek V3 and Kimi K2 series (#14194).

Dependency upgrades: transformers bumped to 5.12.1 (#29393); tvm-ffi / sgl-deep-gemm / tilelang upgraded (#29554).
and . See the DeepSeek-V4 cookbook.

Full release notes by category below.

New Model Support

• Hunyuan 3 (Hy3): #30201 (cookbook)
• Hierarchical Reasoning Model (HRM-Text): #27887
• NVIDIA LocateAnything-3B: #28958
• Baidu Unlimited-OCR: #29186 (cookbook)
• JoyEcho multi-shot A/V: #27420 (cookbook)
• Qwen3.6 ModelOpt mixed NVFP4: #27906, #29905 (cookbook) ⭐

GLM-5.2

• [BCG][GLM5] perf: BCG support and prefill enhancements: #27053 ⭐
• [CI] Add GLM52 NVFP4 MTP B200 tests: #30021
• [cookbook] GLM-5.2 NVFP4 B300: TP8 recipe + 3 strategies: #29557
• [Cookbook] GLM-5.2: tune GB300 NVFP4 recipes + fill benchmarks: #29486
• [cookbook] drop redundant serve flags (GLM-5.2) + fix M3 page-size note: #28731
• [Docs] Add NVFP4 quantization to GLM-5.2 cookbook: #29380
• [DSA][GLM5.2] Index Share for MHA: #29959
• [GLM-5] Tune the threshold of router GEMM: #29470
• [Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step: #29787
• Bypass legacy GLM DSA layer types validation: #29454
• docs: add B200 NVFP4 recipes + benchmarks to GLM-5.2 cookbook: #29674
• docs: add PD disaggregation to GLM-5.2 cookbook playground: #29544
• docs(cookbook): add AMD MI300X/MI325X/MI355X support for GLM-5.2: #28471
• Fuse the DSA (V3.2, GLM-5.x) indexer Q/K paths into single kernels: #27705 ⭐
• glm5.2 on ascend doc (new version): #29828
• Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120: #27397
• Update GLM-5.2 B300 and GB300 NVFP4 cookbook settings: #29466
• Update GLM tests to 5.2 and delete redundant tests: #29686
• [AMD] [GLM5] Guard cuda_runtime.h for ROCm in fused_metadata_copy: #29373
• [AMD] [GLM5] Mark EAGLE verified on MI300X/MI325X (gfx942) in GLM-5.1 cookbook: #29313
• [AMD] [GLM5] GLM-5.1 MXFP4 (MI355X) + enable EAGLE for gfx950 in cookbook: #29194
• [AMD] [GLM5] Add opt-in Triton fp8 sparse-MLA prefill kernel for gfx950: #28975
• [AMD] [GLM5] skip redundant -inf pre-fill of HIP indexer MQA-logits: #28757

DeepSeek V4

• [DeepSeek V4] Enable FlashMLA sparse prefill by default: #29775 ⭐
• [DeepSeek-V4] Add an opt-in non-paged indexer for long-context prefill: #29619 ⭐
• [DSA] Fold page-table into fused top-k v2 (decode): drop page_size=1 expansion: #30274
• [JIT Kernel] DeepSeek-V4 DSA indexer: faster top-k + page-table transform (runtime k <= 2048): #26788 ⭐
• [Cherry-pick to release/v0.5.15] [DSA] Fix IMA in fused top-k v2: write all output slots on tie overflow (#30512): #30559
• [Cherry-pick to release/v0.5.15] [DSV4] perf: Make FP8 quant output tensor contiguous (#27926): #30449
• [Cherry-pick to release/v0.5.15] [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks (#30140): #30436
• [Cherry-pick to release/v0.5.15] [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378): #30427
• [AMD] Improve performance of DSV4 in high concurrency: #28938
• [AMD] DSV4 aiter reduce-scatter decode: #29103
• [AMD][DSV4] Remove per-batch D2H syncs in MTP to avoid bubbles between 2 batches: #29420
• [AMD][DeepSeek V4] Fix default FlashMLA sparse prefill off on ROCm/HIP: #29982
• [AMD] Fix DeepSeek V4 MTP accuracy issue: #30333
• [AMD] Fix dsv4 indexer dtype dispatch on gfx950: #29479
• [AMD] Cap DSV4 Flash max_total_num_tokens: #30313
• [AMD] Fix DeepSeekV4 server cutlass error: #30374
• [Intel GPU] DeepSeek V4 10/N : Add sqrtsoftplus support to fused_topk_torch_native: #28048
• [Intel GPU] DeepSeek V4 7/N: Support fused_rope_inplace on XPU using triton: #27915
• [Intel GPU] DeepSeek V4 3/N: Support hc_split_sinkhorn on XPU using sgl_kernel: #27783

Speculative Decoding

• [MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash): #30079
• [perf] tiny optimize select_index op for draft extend: #29078
• [perf] simplify _apply_cuda_graph_metadata for draft extend in trtllm_mla backend: #29077
• [Spec] Enable FlashInfer autotune for spec draft: #29595
• Speculative decoding support on XPU: #23180
• (perf): Shard Kimi-K2.5 Eagle3 draft fc + symm-mem AG: #29223
• [XPU] Unbreak stage-b: re-add --disable-decode-cuda-graph, quarantine EAGLE3 parity: #30048

Piecewise & Breakable CUDA Graph

• [Experimental] Full Cuda Graph Support for Prefill: #27988 ⭐
• Disable dsr1 prefill cudagraphs by default: #28053
• Enable Breakable Cuda Graph as Default: #29458 ⭐
• [AMD] Enable BCG on ROCm + route aiter prefill via MHA during PCG/BCG capture for Kimi-2.5: #27833
• [XPU] Enable XPU graph support (decode full-graph + prefill tc_piecewise): #29053

Attention Backends

• [GDN][KDA] ReplaySSM buffered output-only decode for Linear Attention: #28451 ⭐
• [KDA] Add FlashKDA prefill backend for safe-gate KDA linear attention: #29472 ⭐
• [KDA-Pilot] Add LTX2 QKNorm split-RoPE CUDA fast path: #29708
• [KDA-Pilot] Add diffusion residual-gate CUDA fast path for LTX2: #29361
• [KDA-Pilot] Add diffusion causal Conv3D cat-pad CUDA fast path for Cosmos3: #29281
• [MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer: #29771
• [NVIDIA] Support flashinfer a2a with flashinfer_trtllm_routed moe: #22394 ⭐
• Re-enable SM90 FlashInfer allreduce fusion with safe backend defaults: #28789
• [CPU] optimize GDN prefill performance: #29117
• [NPU] perf: precompute mamba conv-state track indices once per batch: #29105

MoE & Expert Parallelism

• [JIT Kernel] Triton moe fused gate: #25835
• [MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771): #29997
• [Perf][Kernel] Fuse SiLU+Mul into NVFP4 Expert Quantization for CUTLASS MoE: #18612
• Add GB10 FP8 fused MoE Triton config: #25665
• [AMD] Fuse shared-expert append + DeepEP remap into one Triton kernel: #28450
• [AMD] fix(moe): correct fused shared-expert scaling on aiter/DeepEP path (mori all-to-all): #28237
• [AMD] Implement QuarkW4A8MXFp4MoE to support amd/gpt-oss-120b-w-mxfp4-a-fp8: #27204
• [AMD][MORI-EP] Skip LocalExpertCount kernel in decode graph when not recording: #30302
• [NPU] [DOC] add missing DEEP_NORMAL_MODE_USE_INT8_QUANT for w8a8+deepep scenarios: #29937

Quantization

• [diffusion] Add Qwen-Image ModelOpt NVFP4 support: #28928
• [Kernel] Add SM90 Q8KV8 FP8 Sparse MLA Prefill JIT Kernel with Tests and Benchmark: #25751
• [weight checker] refactor: add precision branch; allow ULP quant err; used chunked compare: #28974
• Add Intel Quantization Support in SGLang: #18139
• Fix gfx95 bpreshuffle FP8 activation scale layout: #29275
• :sparkles: [llm][npu][quant] Add W4A8 MXFP quantization support for Qwen3 Dense on Ascend NPU: #23650
• [AMD] Gate broken CK block-FP8 GEMM shapes to aiter-triton-GEMM to fix ROCm 7.0 Qwen3.5 accuracy: #29918

Parallelism & Disaggregation

• [feature] implement dcp for deepseek_v2: #14194 ⭐
• [HiCache] Add NIXL FILE cache cleaner: #28258
• hisparse: support NIXL DRAM KV destinations for HiSparse: #27563
• [AMD]: Enable NIXL PD disaggregation for ROCm(1/n): #28348
• [AMD]: docker(rocm) bump Mooncake to latest main + enable multi-protocol: #27730
• [AMD] Support triton backend decode context parallel for Qwen3.5: #25090
• [AMD] Fix AITER custom all-gather CUDA-graph capture crash under torch_memory_saver: #30557

Scheduler & Runtime

• [Perf] Overlap result D2H copy with the next forward step: #29075
• [refactor] Add a read-through server_args accessor to RuntimeContext (stack 1/15): #30063
• [scheduler] Add scheduler metrics reporter init hook: #29535
• Add native Exa-backed web_search support: #29342 ⭐
• Add scheduler metrics extension hooks: #29207
• feat(metrics): add Prometheus metrics for the EPD encoder server: #27564
• Support DP-aware PD router dispatch: #26245

HiCache & Radix Cache

• [HiCache] Optimize HiCache hash generation with bulk token byte conversion: #28287
• [HiCache][AMD] Add UMBP tiered DRAM + SSD L3 storage backend with hugepage host allocator: #25377
• [optimize] fix swa eviction boundary for unfinished inserts: #29350
• Optimize C128 state pool allocation using request state pool: #28612 ⭐

Multimodal

• [EPD] Optimize multimodal global cache with paged embedding pool: #28441
• [VLM] Qwen3-VL / Moss-VL ViT preprocessing optimizations: #28940 ⭐
• Add MiMo V2.5 Blackwell vision FA4 recipe: #29253

Model Support & Optimizations

• [NVIDIA] Support TF32 matmul to improve MiniMax gate gemm performance: #22744
• [trtllm_mha] Fuse cuda-graph metadata rebuild into one triton kernel: #29843
• Add fused EH norm for DeepSeek NextN: #29667
• Add Laguna XS.2.1 DFlash support to SGLang: #29446
• Fused QK GemmaRMSNorm + RoPE + gate kernel for Qwen3.5: #28320
• perf(triton): avoid per-step D2H .item() sync in cuda-graph loc translate: #29921
• Support Cutedsl BF16 GEMM JIT kernel: #30117

SGLang-Diffusion

• [diffusion] perf: add unified SP shard helpers and zero-copy tail-pad attention: #30107
• [diffusion] perf: tp-shard every text/image encoder across the full DiT replica (any parallelism): #30086
• [diffusion] feat: add LingBot realtime prompt, KV window, and lazy VAE controls: #30040
• [diffusion] feat: performance_mode=speed enables torch.compile by default: #30016
• [diffusion] feat: add --offload-during-compile to fit max-autotune on tight-memory GPUs: #29862
• [diffusion][cache-dit] support Krea-2 + run-driven has_separate_cfg: #29688
• [diffusion] feat: support cache-dit for Ideogram 4: #29631
• [Diffusion] Add Krea 2 support: #29052
• [diffusion] fix: add profiling support and fix VBench dataset handling in bench_offline_throughput: #27704
• [diffusion] rl: add sleep/wake support for diffusion engine: #22659
• [AMD][diffusion] Disable layernorm torch.compile decorator in eager mode on ROCm to avoid memory-access fault: #29673

AMD / ROCm

• [mori] Add a combine-kwargs hook and use_external_inp_buf plumbing: #29097
• [AMD] Fuse topk padded-token masking into a single Triton kernel: #28084

NPU / Ascend

• [NPU] [DOC] Update arguments detail to NPU support features page: #30328
• [NPU] [DOC] Update deterministic inference feature support status to A2, A3: #29632
• [NPU]GLM-4.7-Flash optimize with fused kernels: #29509
• [NPU] [DOC] Add environment prerequisites to model tutorials: #29293
• [NPU] Support fsdp for rl_on_policy_target: #29128
• [NPU] adapt_fused_rope_qk_mqa_optimize: #28872

CPU / Intel / XPU

• [qwen3.5][XPU]Add XPU support for set_embed_and_head and fused QK RMSNorm kernel: #27870
• [Apple Silicon] Add labeler config: #29908
• [CPU] add fused_qk_gemma_norm and refactor norm kernel implementation: #30216
• [CPU] enable fused_sigmoid_mul on CPU device: #29378
• [Intel GPU] add pytorch profiling support for XPU in bench offline throughput and enhance num steps: #28308
• [XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default: #29911

Dependencies

• [Deps] Bump transformers to 5.12.1: #29393 ⭐
• feat(sgl-kernel): add InfLLM v2 attention kernels: #29383
• Upgrading tvm-ffi/sgl-deep-gemm/tilelang: #29554 ⭐

All PRs included in this release: v0.5.14...v0.5.15

New Contributors

Approximate list from git history; GitHub's "Generate release notes" produces the canonical, deduped list.

• Augusto Yao made their first contribution in #14194
• Wang, Mengni made their first contribution in #18139
• meinie made their first contribution in #21531
• ANSHUMAN TRIPATHY made their first contribution in #23180
• Rohit Harkhani made their first contribution in #25071
• TzZtzt made their first contribution in #25153
• Daniel Stokes made their first contribution in #26255
• Jiajun Li made their first contribution in #26980
• Kaixi made their first contribution in #27053
• Stanley Winata made their first contribution in #27204
• hhhh1252023 made their first contribution in #27433
• Oxana Korzh made their first contribution in #27835
• yifei wu made their first contribution in #27887
• chengcuiping made their first contribution in #28401
• Raiden Makoto made their first contribution in #28455
• Martin Hua made their first contribution in #28481
• Tai An made their first contribution in #28503
• a60124901 made their first contribution in #28586
• Zhihao Wang made their first contribution in #28676
• toufupi made their first contribution in #28770
• Yuankai Chen made their first contribution in #28787
• Anusha Pant made their first contribution in #28952
• Jyothirmai Kottu made their first contribution in #28958
• qyb233 made their first contribution in #28980
• Alex Tumanov made their first contribution in #28996
• hirakunaramuka2 made their first contribution in #29004
• IvanShan177 made their first contribution in #29102
• Shijin Zhang made their first contribution in #29161
• Aditya Kamat made their first contribution in #29186
• jonah-berman made their first contribution in #29342
• Feng Yao made their first contribution in #29350
• Feng Yao made their first contribution in #29352
• cauphe made their first contribution in #29383
• Siming Deng made their first contribution in #29440
• adamkbaranowski made their first contribution in #29446
• bef0rewind made their first contribution in #29570
• SSSunzt made their first contribution in #29571
• Jzz1943 made their first contribution in #29631
• sushil Dubey made their first contribution in #29672
• YukioZzz made their first contribution in #29756
• san-tian made their first contribution in #29798
• Pranjal Shankhdhar made their first contribution in #29843
• NOOB made their first contribution in #30181

Full Changelog: v0.5.14...v0.5.15