GoldenMatch

Added

• Negative evidence on Fellegi-Sunter (type: probabilistic) matchkeys
  (Formulation B, EM-learned): negative_evidence was previously silently
  ignored on probabilistic matchkeys (weighted/exact only), which meant every
  Splink-converted config — exactly one FS matchkey — had no defense against
  the fan-out/homonym snowball (two distinct people sharing name+city merging
  because name evidence dominates). Each NE field now joins train_em as a
  constrained EM-learned dimension contributing log2(m_fired/u_fired) when
  it FIRES (both values present + scorer(a, b) < threshold, strict <) and
  exactly 0 otherwise — the fired-else-zero clamp is what makes it negative
  evidence rather than a regular scored field. NegativeEvidenceField gets a
  new penalty_bits (log2 LLR fixed override, probabilistic-only, abs()
  applied) alongside the existing penalty (still required on weighted/exact,
  now rejected on probabilistic — set penalty_bits instead). Guards: native,
  fused, and the fast-path scorer all decline NE-bearing FS matchkeys (pure
  Python fallback; a future kernel port adds FS_SUPPORTS_NE); the bucket
  backend's slim-projection keep-list was extended so an NE-only field (e.g.
  phone, never a regular matchkey field) survives the default
  GOLDENMATCH_BUCKET_SLIM_PROJECTION. EMResult.validate_for now requires
  match_weights["__ne__<field>"] for every NE field without penalty_bits,
  so a model trained (or a Splink model imported) before this feature fails
  loudly instead of silently scoring NE at weight 0. An unregistered/unknown
  NE scorer on FS fails loud at train/score time (score_field raises on
  unknown scorers; no _NE_BROKEN swallow on FS) — unlike weighted's
  swallow-and-warn fallback, this is intentional. Continuous/Winkler-path
  (train_em_continuous) NE is out of scope and rejected with a clear error.
  Supersedes the deferral in docs/superpowers/specs/2026-05-21-ne-fs-investigation.md
  (Wave D): that investigation judged the Bayesian-factor formulation correct
  but deferred it believing P(disagree_NE | match) needed labeled pairs —
  stale, since EM already estimates match-conditional probabilities for every
  regular FS field without labels, and the same machinery estimates them for
  NE dimensions.

• Splink migration upgrade pass (goldenmatch import-splink SETTINGS.json --upgrade DATA.parquet --model-out MODEL.json, or
  gm.upgrade_splink_conversion(conversion, data) from Python): a data-aware
  pass over a converted Splink config that applies three independent levers —
  term-frequency tables computed from the data (Splink model exports don't
  carry them, so converted tf fields were inert), distance thresholds
  re-derived from measured string lengths (the converter assumes length 10),
  and link/review thresholds calibrated from the blocked-pair score
  distribution. Writes the upgraded config/model to --output/--model-out
  with the faithful baseline alongside as *.baseline.*, plus a
  baseline-vs-upgraded delta table; --splink-clusters / --labels take
  reference cluster mappings (first column id, second cluster_id) for
  agreement / truth F1 measurement. Measured on the wild-config dogfood bench
  (defaults-vs-defaults, pairwise F1 vs truth): real_time_settings/fake_1000
  0.482 → 0.633 (native Splink: 0.601), saved_model_from_demo/fake_1000
  0.677 → 0.766 (Splink: 0.699), model_h50k/historical_50k
  0.707 → 0.740 (Splink: 0.686) — the upgraded conversion beats native
  Splink on all three pairs.

• Fan-out / negative-evidence upgrade lever (fan_out, in the default
  import-splink --upgrade lever set, between distance_thresholds and
  calibration): detects unused identity-grade columns (phone/email/id-named,
  high-cardinality, non-matchkey, non-blocking) whose disagreement contradicts
  pairs the imported model would confidently merge (posterior >= 0.9), and —
  when the contradiction rate clears the risk gate (>= 2%, >= 10 firing
  pairs) — adds the column as negative_evidence with posterior-weighted
  EM-shape weights written into the upgraded model (__ne__<field> entries).
  Also tunes golden_rules.max_cluster_size = max(10, 2 * reference max cluster size) from --labels (preferred) or --splink-clusters, so
  auto_split catches mid-size homonym snowballs the static default (100)
  ignores on person-shaped data. The calibration lever is now NE-aware
  (fs_weight_range + per-pair NE contributions; its warn+skip tripwire is
  removed). Wild-bench: no regressions; model_h50k improved 0.7396 -> 0.7421
  on guard tuning alone.

• Native negative-evidence scoring (goldenmatch-native >= 0.1.15): the
  Rust kernels now score FS negative evidence — score_block_pairs_fs AND the
  fused match_fused_fs, which also gained custom level_thresholds banding
  (full kernel parity). Detection is capability-gated (FS_SUPPORTS_NE,
  FUSED_FS_SUPPORTS_LEVEL_THRESHOLDS), so older wheels keep the pure-Python
  fallback with no behavior change; NE-bearing matchkeys previously always
  took the pure-Python path. The fused path declines derive_from NE (its
  raw-columns entry never materializes derived columns).

Fixed

• Threshold calibration on imported Splink models: the calibration lever
  now re-estimates the within-block match rate from the model's likelihood
  ratios instead of trusting proportion_matched, which on imported models
  holds Splink's probability_two_random_records_match — a random-pair prior
  orders of magnitude below the post-blocking rate the percentile math
  expects. Trusting it cut at the extreme top of the score distribution and
  collapsed recall (F1 0.482 → 0.157 on the bench pair above).
• Tuned max_cluster_size survives the YAML loader round-trip: the
  config loader's golden_rules normalization swept max_cluster_size into
  field_rules (the "set programmatically, never via YAML" assumption went
  stale when the fan-out lever started writing it), so import-splink --upgrade output failed to reload. It is now a recognized top-level
  golden_rules key.