Skip to content

GoldenMatch

v3.6.0 Feature

This release adds 3 notable features for engineering teams evaluating rollout.

Published 6d Data Pipelines & ETL
✓ No known CVEs patched
Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

data-cleaning data-engineering data-matching data-quality deduplication entity-resolution
+14 more
fellegi-sunter fuzzy-matching knowledge-graph llm master-data-management mcp-server polars pprl python record-linkage rust splink typescript zero-config

Summary

AI summary

Zero‑config now routes probabilistic datasets to Fellegi‑Sunter by default.

Full changelog

Changed

  • Zero-config now routes probabilistic-shaped datasets to Fellegi-Sunter
    by default (#1874).
    A dataset with no surviving strong-identity exact
    matchkey (identifier/email/phone) and 2+ fuzzy fields is served by the
    EM-weighted FS path instead of exact+weighted matchkeys -- measured F1
    lifts on error-heavy PII (historical_50k 0.62 -> 0.78) with no regression
    where a strong key survives. Kill-switch:
    GOLDENMATCH_AUTOCONFIG_ROUTE_PROBABILISTIC=0.

  • Config-healer loop cost is now bounded and tunable (#1404). heal (and
    the review_config / suggest_from_result verify path it drives) could run up
    to step_cap × (1 + max_verify) full pipeline passes per call — ~45 on the
    defaults — plus a full goldencheck blocking_risk variant scan (O(distinct²)
    per string column) re-run on the unchanged frame every iteration. Three cost
    levers, all defaulting to byte-identical behavior:

    • The goldencheck variant scan is memoized for the whole heal loop via a
      new variant_risk_cache() scope (core/suggest/adapter.py) keyed on the
      data-column set — it runs once over the frame instead of once per iteration.
      Output is unchanged; only the cost moves.
    • Verify fan-out is tunable: review_config/suggest_from_result/heal
      take max_verify, and GOLDENMATCH_SUGGEST_MAX_VERIFY sets it globally
      (default 8). Since the healer applies only the top surviving suggestion,
      max_verify=1 verifies just that candidate — the cheapest mode.
    • Marginal-gain early-stop: heal(min_health_gain=…) /
      GOLDENMATCH_HEAL_MIN_HEALTH_GAIN stops the loop once cluster-health gain
      flattens (fail-open — a result without clusters never triggers a stop). Off
      by default. GOLDENMATCH_HEAL_STEP_CAP also exposes the outer cap.

Added

  • DedupeResult.identity_summary (#1913). The per-run identity-
    resolution summary (entities created/absorbed/merged) is now surfaced on
    the public result -- None when identity resolution is disabled. Backs the
    in-Postgres gm_resolve write path in goldenmatch-pg.
  • Small-N Fellegi-Sunter routing floor (#1947). Below
    GOLDENMATCH_FS_ROUTE_MIN_ROWS rows (default 500) a probabilistic-shaped
    dataset stays on the robust weighted path -- FS EM is data-starved at small
    N and under-merges fuzzy-close variants. 0 disables the floor. Every
    dataset that validated the FS default is far above it, so routing there is
    unchanged.

Fixed

  • Net-zero-evidence filter kills scale-growing FS over-merge (#1899,
    default ON).
    Pairs whose only agreement is on absent (unobserved) fields
    no longer accrue spurious match weight; ported to the numpy and native
    kernels.
  • FS at 1M rows (#1896). A pairs-budget blocking gate stops a low-
    cardinality pass from compounding into a megablock, and an Arrow pair-
    stream plus EM block-sample cut the FS memory peak.
  • Unobserved record_embedding masked out of the weighted score
    (#1859).
    A record with no embedding no longer contributes max agreement.
  • Arrow-native auto-config blocking profile emitter (#1946). The
    controller's sample iterations no longer force an arrow->polars round trip
    just to count rows, so zero-config runs correctly (no degraded RED-sentinel
    config) on a base, polars-free install. Byte-identical with polars present.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track GoldenMatch

Get notified when new releases ship.

Sign up free

About GoldenMatch

All releases →

Related context

Related tools

Earlier breaking changes

  • v3.1.0 `GOLDENMATCH_FRAME=polars` now requires the `[polars]` extra; raises error without it.
  • vgoldencheck-v3.0.0 `inferred_type` emits neutral dtype vocabulary (str/int/uint/float/date/datetime/bool/other) instead of raw Polars dtypes.
  • vgoldencheck-v3.0.0 'inferred_type' now emits a neutral dtype vocabulary instead of raw Polars dtype strings.
  • vgoldencheck-v3.0.0 `scan_file`, `scan_dataframe`, and CLI `check` now run without Polars, using Arrow-native pyarrow.Table.
  • v3.0.0 Result frames now return pyarrow.Table instead of Polars DataFrame.

Beta — feedback welcome: [email protected]