kael-bit/engram-rs

Recall Overhaul

• IDF-weighted term scoring: Query terms weighted by corpus rarity (ln(N/df)). Rare terms boost relevance more than common ones — works for both keywords and natural language queries.
• Multiplicative scoring: score = relevance × (1 + 0.4×weight + 0.2×recency). Zero relevance → zero score, regardless of memory weight.
• Dynamic noise filtering: Replaced 100+ hardcoded stopwords with df/N > 0.8 auto-detection. Language-agnostic, zero maintenance.
• IDF miss penalty (0.85×) and orphan query penalty (0.5×) for better discrimination.
• Tags as soft boost: Tag params no longer hard-filter. Each match → relevance × 1.2.
• min_score default 0.30 at API layer to filter noise.

Topiary Clustering

• Tag-aware merge/enforce: Combined similarity (cosine × 0.7 + tag_jaccard × 0.3) prevents grouping semantically-near but conceptually-different memories.
• Absorb stays pure cosine: Fragments consolidate regardless of tag differences.
• Absorb threshold 0.40 → 0.30 for easier fragment cleanup.

Prompt Quality

• Unified tag guidance across all 7 LLM entry points: "name the SUBJECT, not meta-properties".
• Strict kind definitions: procedural = reusable step-by-step workflows ONLY.
• SETUP.md templates updated with same guidance.

Tests

• Removed 38 trivial tests (-594 lines). 239 remaining tests all pass.