Reame

v0.1.4 — the first-users release

Every fix in this release came from the project's first days with real users (thanks, HN):

The chat endpoint now speaks each model's chat template. /v1/chat/completions used to concatenate messages as plain role: content text — a format chat models were never trained on — so they never emitted their end-of-turn token and every reply ran to max_tokens, hallucinating both sides of the conversation. Messages now go through the GGUF's own template, and generation stops at any end-of-generation token (ChatML's <|im_end|> included) on all three decode paths: classic, speculative, interleaved. Measured: a short factual answer went from 600 tokens of rambling to 7 tokens and finish_reason: "stop".

One GGUF is enough out of the box. The default config required a second draft model for speculative decoding and died with a bare "No such file or directory" when it was missing. The default is now mode = lookup (n-gram drafting, no second model needed), lookup mode no longer loads a stale draft_model_path, and a missing model file fails fast naming the exact path and how to fix it.

Clear error for a wrong model.path. Relative paths resolve from the working directory — the error now says so. /opt is gone from the examples; models and cache default to project-relative paths.

Interleaved serving: completion state moved under its own mutex. Under load the stepping loop could starve finished() waiters, so cross-thread signals (like the Conclave's early-consensus stop) landed late or never.

Prebuilt binaries below: Linux x64, Linux ARM64, macOS ARM64 (runtime dependency: libzstd).