Skip to content

BigMoeOnEdge

v0.15.1 Feature

This release adds 3 notable features for engineering teams evaluating rollout.

Published 4d LLM Frameworks
✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

android c++ edge-ai gemma gguf gpt-oss
+7 more
inference llama-cpp llm mixture-of-experts moe on-device-ai qwen

Summary

AI summary

Adds warnings when cache‑aware expert dropping operates outside the measured top‑k range.

Full changelog

Ships the warning that 0.15.0 was tagged a few commits too early to include.

Why this release exists

0.15.0 turned cache-aware expert dropping on by default in the app, at 75% of the uniform share. That threshold is a fraction of 1/top-k, so what it discards depends on how wide the model's routing is:

| top-k | uniform share | what 75% means |
|---|---|---|
| 8 | 12.5% | skip below 9.4% — where every published number was measured |
| 4 | 25% | skip below 18.8% |
| 2 | 50% | skip below 37.5% — a miss discards the whole minority expert |

gpt-oss routes 4 experts of 128, and it is reachable from the app. 0.15.0 therefore shipped a default operating well outside its measured range, with nothing in the engine or the UI saying so.

What changed

  • The engine warns once at load when dropping is armed and the effective top-k is 4 or fewer, quoting the real share for the model in hand.
  • The app shows the same caveat inline under the setting, computed from the width the loaded model reports — not assumed, and only once a model is loaded.
  • It warns rather than clamping. The engine cannot know whether that trade is acceptable for a given model and task, and silently adjusting a number the caller chose would be worse than a loud caveat.
  • BMOE_READY gains n_expert_used (effective width after any override, 0 on a non-MoE model); Session::n_expert_used() exposes the same to embedders. Additive — older consumers ignore it.

Also corrects a contradiction inside the 0.15.0 changelog section, where the app-default entry still called the quality cost unquantified while the Measured block below it reported the GSM8K result.

If you are on 0.15.0

Nothing is broken and no output changes. Upgrade to be told when the setting is operating outside the range it was measured in — or set Drop cold experts to off on models that route few experts.

Full changelog: CHANGELOG.md

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track BigMoeOnEdge

Get notified when new releases ship.

Sign up free

About BigMoeOnEdge

All releases →

Beta — feedback welcome: [email protected]