This release adds 2 notable features for engineering teams evaluating rollout.
Published 24d
Data Pipelines & ETL
✓ No known CVEs patched
✓ No known CVEs patched in this version
Topics
data-cleaning
data-engineering
data-matching
data-quality
deduplication
entity-resolution
+14 more
fellegi-sunter
fuzzy-matching
knowledge-graph
llm
master-data-management
mcp-server
polars
pprl
python
record-linkage
rust
splink
typescript
zero-config
Affected surfaces
auth
Summary
AI summaryGroup/list attribute demotion for exact matchkeys is now enabled by default.
Full changelog
Added
- Group/list-attribute demotion for EXACT matchkeys on single-source person data — now DEFAULT ON. A shared GROUP/LIST/FACILITY value — a shared switchboard
phoneline, a mailing-list / campaignidentifier(tl_id), a facility NPI, a roleemailinbox — is not a person-identity claim: as anexactmatchkey (exact_phone,exact_tl_id) it force-merges every DIFFERENT person sharing it into one mega-cluster.autoconfig_discriminative.should_demote_attribute_fielddemotes such an exact matchkey to blocking-only when the LARGE shared-value groups do NOT co-agree on the person name. Design: (1) group-size-aware — co-agreement is measured over large shared-value groups only (≥10 records), so a real personal id that only ever groups a person's few duplicates is KEPT (insufficient support), while a campaign list / facility id / role inbox that groups many different people is demoted — this also stops a mostly-unique column (atl_idat 0.53 cardinality) from being rescued by its many small same-person groups averaging the signal up (measured: big-group name-power 0.01 vs small-group 0.80); (2) person-name basket — co-agreement is measured against person-name columns only, not the broad #1351 identity basket (polluted on real data by constant metadata mis-typed asidentifier).- Scoped to EXACT uses only, deliberately. The accuracy sweep (
scripts/autoconfig_quality) showed that demoting the same attribute from a weighted fuzzy contributor regressed corruption-heavy data: febrl3 F1 0.99→0.86, ncvr_synthetic 0.96→0.95 — its synthetic addresses also collide across people, but with the names corrupted the weighted address field is load-bearing for matching true duplicates. Restricting to exact matchkeys keeps that recall (a soft contributor never force-merges) while still killing the hard force-merges. - Default flipped ON in 2.7.0. The exact-only sweep showed zero F1 change across the whole corpus (febrl3 0.9921→0.9921, ncvr_synthetic 0.9636→0.9636, anchors unchanged — flag-on == flag-off on every dataset), while it fixes real-world group-attribute over-merges: measured on a real MJH dermatology list (19,278 rows), the shared-clinic-
phoneand campaign-tl_idexact matchkeys are both dropped, biggest cluster 70→6, clusters recovered 11,347→16,509. Kill-switch:GOLDENMATCH_ATTRIBUTE_DEMOTION=0restores the pre-2.7.0 behavior. (PRs #1368, #1370.)
- Scoped to EXACT uses only, deliberately. The accuracy sweep (
Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Share this release
About GoldenMatch
All releases →Related context
Related tools
Earlier breaking changes
- v3.1.0 `GOLDENMATCH_FRAME=polars` now requires the `[polars]` extra; raises error without it.
- vgoldencheck-v3.0.0 `inferred_type` emits neutral dtype vocabulary (str/int/uint/float/date/datetime/bool/other) instead of raw Polars dtypes.
- vgoldencheck-v3.0.0 'inferred_type' now emits a neutral dtype vocabulary instead of raw Polars dtype strings.
- vgoldencheck-v3.0.0 `scan_file`, `scan_dataframe`, and CLI `check` now run without Polars, using Arrow-native pyarrow.Table.
- v3.0.0 Result frames now return pyarrow.Table instead of Polars DataFrame.
Beta — feedback welcome: [email protected]