YantrikDB

First batch of engine changes responding to the yantrikdb-agi Phase 1 + Phase 2 gap analysis (2026-05-26 / 2026-05-27). Three engine-side issues + one supporting infrastructure baseline. Tier 1 (validation gate UX, remember template system) was forwarded to yantrikdb-server. Tier 3 (RFC for first-class link model on the remember write path) deferred to v0.8.0 work.

Highlights

#45 — think() redundancy trigger names the rid pair (PR #49)

Phase 1 gap T1 / Phase 2 Proposal 7.1. The redundancy + potential_conflict triggers fired correctly but consumers reading only the reason string saw an opaque "Two memories are X% similar and may be redundant" with no rids — they had to manually search for the duplicate pair via separate recall round-trips.

The source_rids field on the Trigger struct already carried the pair; the user-visible reason now embeds it too. Compact snippet_a / snippet_b / rid_a / rid_b keys added to context so callers can preview both memories without follow-up recall. Full text_a / text_b retained for back-compat.

Both check_redundancy passes patched (sim > 0.85 redundancy + entity-overlap potential_conflict; sim ∈ (0.65, 0.85] substitution-category potential_conflict).

#46 — certainty first-class on remember + recall (PR #50)

Phase 2 Proposal 4. The engine had certainty as a record() param and a field on RecallResult, but it was structurally invisible on the recall path — no filter, no sort, no surface to use it.

• recall() gains certainty_min: Option<f64> — filters candidates with certainty < min BEFORE scoring (saves work).
• recall() gains order: Option<&str> — re-sorts the final top_k after MMR. Accepts \"relevance\" (default) | \"certainty\" | \"recency\". Unknown values surface as YantrikDbError::InvalidInput.
• New RecallOrder enum + parse_recall_order() helper.
• RecallQuery builder gains .certainty_min(f64) and .order(&str).
• Python binding threads both params with Pythonic defaults.

Candidate retrieval still uses HNSW + MMR diversity. order only re-sorts the final selected pool — production-honest semantics that don't bypass approximate-NN.

#47 — correct() in-place mutation + revision history (PR #51) — BREAKING CHANGE

Phase 2 Proposal 6. The deepest legit ask in the algo gap analysis. The v0.7.19 correct() minted a new rid + tombstoned the original — every inbound reference (graph edges, replication audit rows, manual supersedes metadata) dangled afterwards, and the only audit trail was a single metadata.correction_note string easily missed.

v0.7.20 correct():

• Mutates in place — preserves rid + created_at
• Appends a row to record_revisions capturing the prior state
• Leaves inbound link integrity intact (anything pointing TO this rid continues to resolve)
• Atomic — revision INSERT + memories UPDATE in one transaction

Engine signature (BREAKING):

pub fn correct(
    rid: &str,
    new_text: Option<&str>,
    metadata_merge: Option<&serde_json::Value>,
    new_importance: Option<f64>,
    new_valence: Option<f64>,
    reason: &str,                  // REQUIRED, non-empty
) -> Result<CorrectionResult>

new_embedding REMOVED (HNSW doesn't support in-place vector update; embedding changes still use forget+record). correction_note REMOVED — replaced by reason as required positional. metadata_merge NEW (patch-merged into existing metadata).

Python binding:

db.correct(rid, reason, new_text=None, metadata_merge=None,
           new_importance=None, new_valence=None)

New db.history(rid) exposed — returns revisions oldest-first.

Schema migration v30 adds record_revisions table:

CREATE TABLE record_revisions (
    revision_id      TEXT PRIMARY KEY,
    rid              TEXT NOT NULL,
    revision_num     INTEGER NOT NULL,
    prior_text       TEXT NOT NULL,    -- encrypted if provider attached
    prior_metadata   TEXT NOT NULL,    -- encrypted if provider attached
    prior_importance REAL NOT NULL,
    prior_valence    REAL NOT NULL,
    reason           TEXT NOT NULL,
    applied_at       REAL NOT NULL,
    hlc              BLOB NOT NULL,
    origin_actor     TEXT NOT NULL,
    UNIQUE(rid, revision_num)
);

UNIQUE(rid, revision_num) + INSERT OR IGNORE makes replication apply idempotent across re-syncs.

Cross-version compat: v0.7.19 followers receiving a v0.7.20 leader's correct op silently no-op (the new payload has no "rid" key under the old shape). Rolling forward in-order is required; rolling back v0.7.20 → v0.7.19 AFTER correct() calls have landed is unsupported — the in-place mutation cannot be reversed automatically.

Supporting infrastructure

• First committed perf-at-scale baseline under docs/benchmarks/perf_at_scale_2026-05-21.{md,json} (run on AMD Ryzen 9 5950X, dim=384, hash-based embeddings, Gram-Schmidt cosine-similarity-targeted perturbation). recall@10 = 1.000 across N ∈ {100, 1000, 2000, 5000}; recall latency scales sub-linearly with N.
• engine_bench harness fix — pre-v0.7.18 bench harness was structurally broken (no spawn_all_workers, no Backpressure retry). Patched in 859c3a8.

Migration / upgrade notes

• Existing DBs: schema v29 → v30 migration runs on first open. Forward-only; no data loss.
• Engine API callers: correct() signature changed (see above). Python db.correct(rid, reason, ...) is now positional reason after rid.
• Embedding-level corrections: continue to use forget() + record(); correct() only supports text + metadata + importance + valence.

Coordinated work

• yantrikdb-server notified of #47 breaking signature change via swarm — their Tier 1 validation-UX work (Proposal 5 + Proposal 8) builds on this PR's correct() semantics. They'll bump engine dep in their next release ceremony.
• yantrikdb-agi notified of issue triage + linked back to the source documents (launch/_YDB_GAP_ANALYSIS_2026-05-26.md, launch/_YDB_PHASE2_PROPOSAL_2026-05-27.md).

Stats

1516/1516 lib tests pass on Windows. CI green on Linux x64 + macOS-14 + Windows + Lint+Clippy+Rustfmt for each PR before merge. Fmt clean across the entire workspace.

What's next

• #48 RFC for first-class link model on the remember write path (Tier 3 from the gap analysis triage) — pure document work, v0.8.0 territory.
• Postmortem on trader's pre-v0.7.19 23k orphan source (sync_peers=0 ruled out replication; suspects = pre-oplog migration data / yantrikdb import CLI / pyo3-direct path).

🤖 Generated with Claude Code