YantrikDB

Hotfix for v0.7.0's write throughput regression diagnosed via yantrikdb-server cross-validation (8w/4r/30s/dim=384 bench).

TL;DR

| Metric | v0.6.7 | v0.7.0 | v0.7.1 |
|---|---|---|---|
| Write tput | 1000/s | 156/s | 1055/s |
| Write p50 | 3.25ms | 39.9ms | 3.8ms |
| Read p99 | 971ms | 335ms | (unchanged from v0.7.0) |
| Engine pressure | 100% | 11.7% | 0% |

Write throughput recovers to v0.6.7 levels. The v0.7.0 read p99 fix stays — different mechanism, untouched by this hotfix.

What was wrong

v0.7.0 wired Phase 4.3's log_op_pending into foreground record() and record_with_rid(). That exposed a latent inefficiency: log_op_pending's backpressure check did a per-call SELECT COUNT(*) FROM oplog WHERE applied = 0, requiring two Mutex<Connection> acquisitions (read_conn for the count, conn for the insert) and an index scan whose cost grew with pending-row count.

At 8 writers, that's ~16 conn acquisitions/sec just for the backpressure check, on top of the actual SQL work.

The fix

Replace the SQL count with an in-memory AtomicI64 counter on YantrikDB:

• open() initializes the atomic from a one-time SQL count at boot.
• log_op_pending increments via fetch_add(1) after a successful insert (gated on conn.changes() > 0 so INSERT-OR-IGNORE replays don't double-count).
• mark_op_applied decrements via fetch_sub(1) on the apply-win transition (gated on the bool it already returns from the v0.6.6 race-safety fix).

Foreground hot path becomes a single Relaxed atomic load.

count_pending_ops() now returns the cached atomic instead of running SQL. SQL-backed count_pending_ops_sql() is kept as a reconciliation oracle for debugging.

Migration

Zero migration. No schema change, no API break. Just bump:

yantrikdb = "0.7.1"

Tests

1416 lib tests pass. Two test updates required (both use the public mark_op_applied / log_op_pending APIs instead of raw SQL, which is the right shape anyway).

Architecture lesson

Phase 4.3's SQL-off-foreground architecture is sound (proven by the v0.7.0 read p99 win staying in place). The regression came from not auditing log_op_pending itself when it got wired into foreground. v0.7.1 audits and fixes; the architecture stays.

Diagnosis credit

yantrikdb-server msg b951a2de — running v0.7.0 cross-validation, surfacing the 5× tput drop with clear empirical signal that let us identify the SELECT COUNT pattern as the bottleneck.