Skip to content

BigMoeOnEdge

v0.13.0 Feature

This release adds 1 notable feature for engineering teams evaluating rollout.

Published 7d LLM Frameworks
✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

android c++ edge-ai gemma gguf gpt-oss
+7 more
inference llama-cpp llm mixture-of-experts moe on-device-ai qwen

Summary

AI summary

Engine now reports how each model honors the Thinking flag with three outcomes: template, prefill, or none.

Changes in this release

Refactor Low

Engine now renders template at load to determine if model honors the "Thinking" flag.

Engine now renders template at load to determine if model honors the "Thinking" flag.

Source: llm_adapter@2026-07-20

Confidence: medium

Full changelog

Turning Thinking off used to set one template variable and hope. That variable is only a request
to the model's chat template, and plenty of templates never read it — so on those models the setting
was dropped in silence and the model reasoned anyway.

The engine now renders the template at load, works out what that model will actually honour, and
says so. Nothing about this is keyed on model names.

Three outcomes, decided from the model itself

  • template — the chat template reads the flag. Nothing else needed (Qwen3, Qwen3.6).
  • prefill — it does not, but reasoning is a structural section of the format, so the turn starts
    past it. Not something the model can decline (gpt-oss / harmony).
  • none — the model declares its own reasoning span and simply cannot be asked to skip it. Reported
    rather than papered over: the app disables the Thinking switch and explains why (LFM2.5).

Where the turn does start past the reasoning, the span is written by llama.cpp's own handler for
that template, so no family's markers appear in this engine at all. That retires the last
model-specific constants in the decode path: priming gpt-oss used to be a literal <|start|>assistant
suffix test plus a literal <|channel|>final<|message|> in session.cpp. An upstream bump that
changes those markers now needs no engine change.

Measured, not assumed. Handing a model that owns its <think> span a pre-closed empty one is a
suggestion it can ignore — and LFM2.5 does, reasoning untagged into the answer, worse than leaving
the setting alone. So the engine does not do it. Verified on-device across the catalog: LFM2.5
reports none and keeps its reasoning tagged and collapsible; gpt-oss-120b and Qwen3.6-35B-A3B
answer with no reasoning at all.

Protocol: BMOE_READY gains think_ctl (template | prefill | none) — see docs/telemetry.md.

Streaming, benchmarks and published numbers are unchanged.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track BigMoeOnEdge

Get notified when new releases ship.

Sign up free

About BigMoeOnEdge

All releases →

Beta — feedback welcome: [email protected]