This release adds 3 notable features for engineering teams evaluating rollout.
✓ No known CVEs patched in this version
Topics
+7 more
Summary
AI summaryUpdates Highlights, CHANGELOG.md, and arm64-v8a across a mixed release.
Changes in this release
| Type | Severity | Summary | CVE |
|---|---|---|---|
| Feature | Medium |
Adds MoE expert-selective streaming for qwen3moe, qwen2moe, and llada-moe models. Adds MoE expert-selective streaming for qwen3moe, qwen2moe, and llada-moe models. Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Adds optional LRU cache and parallel read pool for streamed experts. Adds optional LRU cache and parallel read pool for streamed experts. Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Adds `bmoe-cli` host tool with machine telemetry and CSV sink. Adds `bmoe-cli` host tool with machine telemetry and CSV sink. Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Adds Android example app with live telemetry panel (tok/s, compute‑vs‑flash I/O split, cache hit rate). Adds Android example app with live telemetry panel (tok/s, compute‑vs‑flash I/O split, cache hit rate). Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Provides debug‑signed `app-debug.apk` (arm64-v8a) for on‑device validation. Provides debug‑signed `app-debug.apk` (arm64-v8a) for on‑device validation. Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Ensures lossless operation — byte‑identical results to full in‑memory runs. Ensures lossless operation — byte‑identical results to full in‑memory runs. Source: llm_adapter@2026-07-19 Confidence: high |
— |
| Feature | Medium |
Uses llama.cpp public API with zero fork; relies on eval‑callback and gguf accessors. Uses llama.cpp public API with zero fork; relies on eval‑callback and gguf accessors. Source: llm_adapter@2026-07-19 Confidence: low |
— |
| Feature | Low |
Integrates with llama.cpp using its public API, with no fork of the codebase. Integrates with llama.cpp using its public API, with no fork of the codebase. Source: granite4.1:30b@2026-07-19-audit Confidence: low |
— |
Full changelog
First tagged release of BigMoeOnEdge — a ports-and-adapters engine that streams MoE experts from flash so models larger than device RAM run on-device, built on top of llama.cpp's public API with no fork.
Highlights
- MoE expert-selective streaming for
qwen3moe(Qwen3-30B-A3B and siblings),qwen2moe, andllada-moe: only the routed experts per token are read from flash, with an optional LRU cache and a parallel read pool. - Lossless — byte-identical to a full in-memory run, proven by the synthetic gates and confirmed on a real 64-expert 4 GiB model.
- Zero-fork llama.cpp: streaming rides entirely on the public eval-callback and gguf accessors;
third_party/llama.cppstays a stock upstream submodule. bmoe-clihost tool with machine telemetry and a CSV sink.- Android example app with a live telemetry panel (tok/s, compute-vs-flash-I/O split, cache hit rate).
Assets
The attached app-debug.apk (arm64-v8a) is built by CI and debug-signed for on-device validation, not Play distribution. Push a .gguf to the app's files dir and pick it in the model picker. See examples/android/README.md.
Full details in CHANGELOG.md.
Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Share this release
About BigMoeOnEdge
All releases →Related context
Related tools
Beta — feedback welcome: [email protected]