This release adds 1 notable feature for engineering teams evaluating rollout.
Published 1mo
Developer Productivity
✓ No known CVEs patched
✓ No known CVEs patched in this version
Topics
ai-coding-assistant
archetype
ast-analysis
claude-code
claude-plugin
code-conventions
+12 more
code-review
code-style
developer-tools
javascript
linter
mcp
mcp-server
prompt-engineering
python
ruby
rails
typescript
Summary
AI summaryFixed misclassification of Ruby test commands in the exec‑log classifier.
Full changelog
The effectiveness eval harness (arc 2): a repeatable A/B measurement of whether chameleon improves agent output, plus one runtime fix it surfaced. First measurement run (24 cells, $8.58, zero errors) is seeded as the tier-ci baseline.
Added
- Effectiveness eval harness (
tests/effectiveness/, local-only, never CI). Realclaude -psessions run identical task prompts under matched arms (off / shadow / enforce, plus--toggle <enforcement key>paired arms for feature-level experiments), in per-cell git worktrees with bootstrap-once profile cloning. Deterministic-first scoring (convention lint, crossfile callers-updated from the calls index, duplication body-hash, verification from the exec log) under a strict metrics-or-unscored contract; a blind pairwise judge panel for the subjective remainder; run.json / run.md scoreboards with committed baselines and a direction-aware regression banner. Two committed convention-rich fixtures (TS + Rails) carry 8 ci tasks; tier-full task packs target the env-pointed real repos. Seetests/effectiveness/README.md.
Fixed
ruby -Itestwas never classified as a test command. The exec-log classifier's\b(?:-Itest|minitest)\bhad an unreachable first branch (no word boundary between whitespace and-), so the standard minitest invocation never recordedtest_command_seenand the Stop-gate test nudge fired even after a passing suite. The pattern now uses a lookbehind, with negative cases pinned.
Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Share this release
About Chameleon
All releases →Beta — feedback welcome: [email protected]