Release history
GoldenMatch releases
All releases
150 shown
Routine maintenance and dependency updates.
Routine maintenance and dependency updates.
Routine maintenance and dependency updates.
Routine maintenance and dependency updates.
Routine maintenance and dependency updates.
- If previously relying on Polars for default scanning, install the `[polars]` extra or switch to Arrow‑native APIs.
- For performance‑critical workloads, consider installing the `[native]` extra to add the compiled kernel accelerator.
- `scan_file`, `scan_dataframe`, and CLI `check` now operate without Polars, using a pyarrow.Table scan frame and fused Rust/Arrow kernels (fallback to pyarrow.compute when native kernel absent).
- `pyarrow` is promoted from an optional to a BASE dependency; `polars` is removed from the base install and moved to the `[baseline]` extra for scipy stat/drift/correlation features and the `[polars]` extra for `scan_dataframe(pl.DataFrame)` convenience overload.
- `inferred_type` output changes from raw Polars dtype strings to a neutral vocabulary (str/int/uint/float/date/datetime/bool/other).
- `pyarrow` becomes a required base dependency enabling Arrow‑native scanning.
- Optional extras `[baseline]` and `[polars]` provide scipy statistics/drift/correlation (using Polars) and DataFrame convenience overloads respectively.
Full changelog
The Flip: the default scan path is now Arrow-native and Polars-free.
Breaking
scan_file/scan_dataframe/ the CLIcheckrun WITHOUT Polars. The scan frame is a pyarrow.Table; profiling routes through the fused Rust/Arrow kernels (with a pyarrow.compute fallback when the native kernel is absent).pyarrowis now a BASE dependency.polarsis NOT: it moved to the[baseline]extra (the opt-in scipy stat/drift/correlation subsystems still use it) and the[polars]extra (thescan_dataframe(pl.DataFrame)convenience overload).inferred_typenow emits a neutral dtype vocabulary (str/int/uint/float/date/datetime/bool/other) instead of raw Polars dtype strings.- Sampling is now an owned deterministic sample (was the Polars-PRNG
sample(seed=42)), stable across runs and workers.
Contract
Non-stat findings are byte-identical to the 2.x Polars path (verified by a finding-set differential: strict Jaccard 1.000 across a 9-dataset corpus, 0 kernel-bug divergences). The only intended differences are the neutral dtype strings and the owned sample.
Install
pip install goldencheck-- Arrow-native scanning, no Polars.pip install goldencheck[native]-- adds the compiled kernel accelerator.pip install goldencheck[baseline]-- adds the scipy + Polars stat/drift/correlation features.
See packages/python/goldencheck/CHANGELOG.md for details.
- Add migration conversion e.g., `pl.from_arrow(result.golden)` where Polars APIs were previously used.
- New env variables for fused routing: GOLDENMATCH_GOLDEN_FUSED, GOLDENMATCH_MATCH_FUSED, GOLDENMATCH_FUSED_PRESSURE_FRACTION, GOLDENMATCH_FUSED_RSS_SCALE, GOLDENMATCH_FUSED_BYTES_PER_PAIR, GOLDENMATCH_FUSED_BYTES_PER_CELL, GOLDENMATCH_FUSED_BLOCK_CONCURRENCY.
- DedupeResult.golden / .dupes / .unique and MatchResult.matched / .unmatched now return pyarrow.Table instead of polars.DataFrame (Polars-eviction W5). Migration requires a one‑liner conversion e.g., pl.from_arrow(result.golden).
- Experimental GOLDENMATCH_FRAME=arrow file ingest lane with Polars‑parity reader corpus and frame‑backend harness.
- Fused auto‑routing at golden and match seams: default live routing of covered slow‑path runs to Arrow‑native kernels (golden_fused) achieving ~2× lower peak RSS, controlled via env knobs GOLDENMATCH_GOLDEN_FUSED, GOLDENMATCH_MATCH_FUSED, etc.
- Standalone fused golden‑record kernel (golden_fused) providing byte‑identical output at reduced memory usage; requires goldenmatch-native 0.1.13.
Full changelog
Changed
- BREAKING: result frames are now
pyarrow.Table.DedupeResult.golden/
.dupes/.uniqueandMatchResult.matched/.unmatchedreturn
pa.Tableinstead ofpl.DataFrame(Polars-eviction W5, spec
docs/superpowers/specs/2026-07-09-goldenmatch-polars-eviction-design.md).
Migration is a one-liner:pl.from_arrow(result.golden). Inputs are
unchanged (polars/pandas/arrow all accepted).to_csvand the notebook
_repr_html_render from Arrow natively. - The arrow ingest lane runs the expression stages (row-ids, standardize,
exact matchkeys) eagerly on the Frame seam; validation rules and auto-fix
are seam-routed with probed arrow twins (RE2 regex owned contract).
Added
- Experimental GOLDENMATCH_FRAME=arrow lane: file ingest via pyarrow with a polars-parity reader corpus and a frame-backend differential harness (Polars-eviction W1).
- Fused auto-routing at the golden + match seams (controller-driven). The pipeline now auto-routes covered slow-path runs to the fused Arrow-native kernels, byte-identically, with no config change and per-surface kill-switches. Golden routing is the broad, LIVE win: at the golden seam the pipeline tries
golden_fused.run_golden_fused_arrowby default on every covered slow-path config and reaches golden output at roughly 2x lower peak RSS; it declines to the classic builder (unchanged output) when the config is uncovered, the native kernel is absent, fullClusterProvenancelineage is requested, or the run is fast-path-eligible. Surfaced onDedupeResult.golden_fused_used; kill-switchGOLDENMATCH_GOLDEN_FUSED=0. Match routing is a documented DORMANT capacity-survival scaffold, not a live default. A controller post-step (maybe_route_fused_match) can short-circuit the match stage to the fused match kernel under an estimated-peak-RSS pressure gate, but the gate is deliberately narrow (auto_split=False, no identity/adaptive/memory/llm_boost/confidence_majority/full-provenance, not across-files-only, a covered weighted matchkey, and real memory pressure). Because zero-config auto-config commitsauto_split=Trueand explicit configs bypass the controller, it effectively never fires by default. When it does fire it is a capacity mode that shedsscored_pairs, cluster-confidence, and lineage to survive; markedDedupeResult.match_fused_capacity_modeand controller-telemetryrule_name+fused_match_post_step. Kill-switchGOLDENMATCH_MATCH_FUSED=0; the wake-up path (a postflight-threshold fast-follow, or an explicit opt-in) is a follow-up. New env knobs (core/fused_routing.py,ExecutionPlan.use_fused_match):GOLDENMATCH_GOLDEN_FUSED,GOLDENMATCH_MATCH_FUSED,GOLDENMATCH_FUSED_PRESSURE_FRACTION(default 0.65), and the est-RSS calibration coefficientsGOLDENMATCH_FUSED_RSS_SCALE/_BYTES_PER_PAIR/_BYTES_PER_CELL/_BLOCK_CONCURRENCY. No new native symbols (reuses thegolden_fused/match_fusedkernels), no new MCP tools / CLI commands / A2A skills. - Fused golden-record kernel (
golden_fused) - standalone, Arrow-native. A new one-FFI-call survivorship kernel (goldenmatch.core.golden_fused.run_golden_fused_arrow, gated bygolden_fused_ready) builds golden records byte-identically to the classicbuild_golden_records_batchpath at ~2x lower peak RSS (measured viascripts/bench_golden_fused_memcap.py+ thebench-golden-fused-memcapworkflow). It covers every Rust-portable survivorship rule in a single native call: scalar strategies (majority / unanimous / first_non_null / longest / source_priority / most_recent / most_complete), quality-weight tie-breaks,field_groupscorrelated survivorship, conditionalfield_rules(predicate AST lowered to a kernel RPN IR viagolden_fused_predicate.lower_predicate/predicate_lowerable),cluster_overrides, andconfidence_majority, plus per-fieldsource_row_idprovenance. Configs that need row-level validators, Python plugins, or LLM survivorship are declined (golden_fused_readyreturns False) and fall through to the classic path unchanged. Not yet wired into the pipeline - this ships as a standalone, benched kernel (composability + peak-RSS win), so no existing user-facing behavior changes. Requiresgoldenmatch-native0.1.13 (the depended-ongolden_fusedsymbol ships in that wheel); without a republished wheel,pip install goldenmatch[native]users transparently degrade to the classic golden path (not a correctness bug - byte-identical output either way). - MCP naming aliases for cross-language parity. The Python and TypeScript MCP servers previously exposed the same operations under different names (
find_duplicates/dedupe,match_record/match,explain_match/explain_pair,profile_data/profile, plus TS'sexplain_cluster). Both servers now answer to both names via non-breaking aliases, so an agent trained against either server can call the other. The Python server gainsdedupe/match/explain_pair/profile/explain_cluster; the TypeScript server gainsfind_duplicates/match_record/explain_match/profile_data. The API-parity gate enforces the nine names staysharedinparity/goldenmatch.yaml. Aliases are excluded from thegoldensuite-mcpaggregated surface so the suite'sprofilestill resolves to goldencheck's file-profiler.
Polars optional for CSV/full scans
- If you use transform_df(pl.DataFrame), read_file, or any Polars objects, migrate by installing with `pip install goldenflow[polars]`. Prefer the new Polars‑free API `transform(data, config)`.
- Polars is no longer a base dependency; it has been moved to the [polars] extra. Pip install goldenflow now runs without Polars by default.
Full changelog
BREAKING: Polars is no longer a base dependency. pip install goldenflow now runs Polars-free by default on goldenflow-native (a base dep); Polars moves to the [polars] extra. All 113 transforms + CSV/Parquet/Excel/DB read + zero-config run without Polars. Migration: pip install goldenflow[polars] if you use transform_df(pl.DataFrame) / read_file / Polars objects; prefer the Polars-free transform(data, config). See CHANGELOG 2.0.0.
goldenflow & goldenflow-native version bumps
Routine maintenance and dependency updates.
- Run the identity migration (`goldenmatch identity migrate-ids --path` or `--dsn`) before upgrading to v2.
- No longer set `GOLDENMATCH_CLUSTER_FRAMES_OUT`; clustering will use Arrow frames‑out automatically.
- Removed Identity `:hash:` scheme; `GOLDENMATCH_IDENTITY_ID_SCHEME` flag and legacy lookup candidate are gone. Persisted DBs with `:hash:` keys will split on next run; must run `goldenmatch identity migrate-ids --path` or `--dsn` before upgrading.
- Removed `GOLDENMATCH_CLUSTER_FRAMES_OUT` environment variable. Arrow frames-out path is now the only clustering path (output‑equivalent). Public `build_clusters` retained as a frames‑backed adapter.
- Removed `RunHistory.cheapest_healthy()` method; replace with `pick_committed()`.
- Stable Sail IdentityGraph API added.
- Identity migration command `migrate-ids` introduced.
- Fellegi‑Sunter block scoring performance improved (~3.5x faster on tiny‑block shapes).
Full changelog
GoldenMatch 2.0.0 -- first backwards-incompatible major.
This release removes four deprecation-window items that shipped with a
1.x runway. The pipeline behavior is output-equivalent; the breaks are in
removed escape hatches, internal shims, and the legacy identity id scheme.
BREAKING CHANGES
- Identity
:hash:scheme removed. The legacy:hash:lookup candidate
andGOLDENMATCH_IDENTITY_ID_SCHEMEare gone. A persisted identity DB
still holding:hash:-keyed records will SPLIT on the next run.
Rungoldenmatch identity migrate-ids --path <db>(or--dsn) BEFORE
upgrading. Un-fingerprintable rows keep their:hash:id. GOLDENMATCH_CLUSTER_FRAMES_OUTremoved. The Arrow frames-out path is
the only clustering path now (output-equivalent). Publicbuild_clusters
is preserved as a frames-backed adapter.RunHistory.cheapest_healthy()removed -- usepick_committed()._scale_aware_backend(internal) removed -- backend selection routes
through the v3 planner.
Migration
If you persist an identity DB, run the migration before upgrading:
goldenmatch identity migrate-ids --path <db> # or --dsn <postgres-dsn>
goldenmatch identity migrate-ids --path <db> --dry-run # counts only
Full guide: https://docs.bensevern.dev/ (Guides -> Migrating to v2).
See packages/python/goldenmatch/CHANGELOG.md for the complete 2.0.0 notes
(Added: stable Sail IdentityGraph API, identity migrate-ids; Performance:
Fellegi-Sunter block scoring ~3.5x faster on tiny-block shapes).
Native-direct SQL graph UDFs + embedding
Routine maintenance and dependency updates.
Routine maintenance and dependency updates.
Auto‑config, checks, classifiers, options, edge
Quality scoring, data‑driven strategies, LLM auto‑enable