This release adds 3 notable features for engineering teams evaluating rollout.
✓ No known CVEs patched in this version
Topics
+13 more
Summary
AI summary_LOKI_RUN_START_SHA now enables isolated base‑suite testing of new/changed tests, LIVENESS adds a 300 s timeout safeguard, and honest scopes provenance to in‑repo test runners.
Full changelog
Accuracy: reverse-classical test provenance (a green test suite must actually prove the change)
The strongest completion signal is "tests pass" -- but a test written to pass proves nothing.
Adopting the FrontierCode insight (grade like a tech lead, not a CI): a NEW test only counts as
affirmative evidence if it FAILS on the pre-change base and PASSES on HEAD. Tests that pass on
both are tautological.
- In council_evidence_gate, when a green test suite is an affirmative candidate (it ran and
passed) and the run has a resolvable base (_LOKI_RUN_START_SHA), loki now runs the run's
NEW/CHANGED test files against the base in an ISOLATED git worktree (never touching the live
tree). If the full suite passes on the OLD code too, the new tests added no failing signal =
tautological, and the test signal is DOWNGRADED from affirmative to inconclusive. - SAFETY (the invariant, verified end-to-end): this can only DOWNGRADE unearned affirmative to
inconclusive. It NEVER fabricates a "confirmed" (a base-suite failure is not attributed to the
new test -> treated as unknown/no-op, never a false-positive), and inconclusive is pass-through
so it NEVER blocks. A legitimate refactor whose regression test passes on base too, with a
non-empty diff, still completes (proven by driving the real gate). No-op on greenfield (no base
or no test files) and when LOKI_TEST_PROVENANCE=0. - LIVENESS: the base run is always bounded (timeout 300s). If no timeout binary is on PATH
(stock macOS without coreutils), the base suite is NOT run unbounded -- provenance skips and
returns unknown (a safe no-op), so a hanging base suite can never stall the completion-time
evidence gate. This can never cause false-green or false-block. - SCOPE (honest): the base worktree is a bare checkout with no dependency install, so in practice
the check fires for in-repo test runners (e.g. pytest against repo sources); ecosystems that
need an install step on base map to unknown -> safe no-op. Coverage grows as the base-materialize
step gains optional dep provisioning. - Records the provenance verdict to .loki/state/test-provenance.json + a trust event.
- Adversarial regression coverage: tests/test-test-provenance-gate.sh covers a genuine fix
(fails-on-base), a tautological test (downgraded), a broken-import base failure (harmless
unknown, not a fabricated confirmed), a refactor that must still complete (real gate pass-
through), the no-timeout-binary liveness path (base run skipped), and the greenfield/opt-out
no-ops.
Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Share this release
About Loki Mode
Multi-agent autonomous SDLC framework. Spec to deployed app. PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief. 5 AI providers, 11 quality gates.
Related context
Related tools
Beta — feedback welcome: [email protected]