v1.1.3 — Tideweaver + T1–T11 curriculum + host-aware rate limits

⚠️ Migration notes — two behaviour changes

If you're upgrading from v1.1.1, two defaults shifted. Both are bug-fix-shaped (the new behaviour is the safer one), but worth checking against your pipelines:

1. fetch_concurrent_payloads no longer aborts the batch on non-429 HTTP errors. Previously, a 503/504 from one source cancelled every sibling in the gather and surfaced as a confusing cancel cascade. Now non-429 errors route to failed_sources like the 429 path always did. Match: existing LoggedIncorporator + get_error DLQ pattern.

2. Host-aware rate-limit defaults for three known-strict APIs. When you don't pass requests_per_second, the engine auto-throttles:

   • api.coingecko.com → 0.2 req/sec (12/min — under the 5–15/min public free tier)
   • pokeapi.co → 1.5 req/sec (90/min — under the 100/min ceiling)
   • vpic.nhtsa.dot.gov → 1.5 req/sec (method-agnostic; covers both GET and POST)

   Caller-supplied requests_per_second always wins; unknown hosts keep the 15 req/sec default. An INFO log line names the applied rate when the registry fires. Impact: a 10-source CoinGecko drill that previously ran in ~700 ms now takes ~50 s — which is what was needed to fit inside CG's per-minute window in the first place.

✨ Headline additions

• Tideweaver orchestration layer. Graph-based coordination over stream() / fjord-flush / export(). Build a Watershed (time window + named Current nodes + dependency edges), hand it to Tideweaver, run. Four shape constructors (parallel / chain / fanout / diamond) + custom with explicit edges. Hard/soft dep gating, skip-ahead, graceful drain at window close, per-current on_error policy. Capstone tutorial at examples/11-tideweaver/.
• Curriculum renumbered to T1–T11. Four formerly-appendix tutorials promoted into the main learning path so the curriculum is one coherent sequence instead of two parallel tracks. Detour interleaving baked into the sequence so CoinGecko's per-minute rate-limit window has time to refresh between CG-heavy steps.
• incorporator tideweaver run|validate CLI with declarative watershed.json config — same env-var interpolation + token-resolution pipeline as stream / fjord. incorporator init --type tideweaver scaffolds the pair.
• Opt-in depends_on graph on fjord seed entries — declare which peer classes a source's inflow(state) reads. When any entry declares it, the seed runs in topological tiers (parallel within tier; sequential between). Default behaviour (no depends_on anywhere) is bit-identical to v1.1.1.
• Typeless-format reads auto-coerce via _schema_union. When a class has been incorp'd from a typed source (JSON / NDJSON / Parquet / SQLite / Avro) and is then read from a typeless format (CSV / TSV / PSV), build_instances() synthesises inc() converters for fields not in conv_dict. CSV round-trips now preserve types automatically.

🩺 Diagnostic ergonomics

• Clearer Seed Error wave from fjord(). KeyError raised inside inflow(state) now produces an actionable message naming the source and the missing peer, with a state.get(...) / depends_on=[...] fix suggestion. No more ["Seed Error: 'Track'"] mysteries.
• Bare-class data-loss warning at outflow flush. A user-pre-declared class Race(Incorporator): pass with no extra fields silently dropped every row field under Pydantic V2's default extra='ignore'. Now emits a one-time WARNING per class identity with a concrete fix.
• analyze_error() inspector survives cp1252 stdout. The error inspector's emoji prefixes no longer crash on Windows consoles — a local print helper catches UnicodeEncodeError and falls back to ASCII.

🚀 Performance + refactor wins

• Cache, DSA, and DRY quick wins: Counter for O(n) duplicate check in Watershed._validate_graph, id()-map for O(n) sequential seed, LRU cache on infer_format / infer_compression, bumped _get_cached_adapter LRU from 128 → 4096, lazy _transitive_upstreams cache on immutable Tideweaver topology, _extract_results_array DRY helper across paginators.
• Async DNS in SSRF redirect hook — _host_is_internal no longer blocks the event loop on DNS.
• Explicit per-subclass container allow-list replaces the older dir(base_class) scan in infer_dynamic_schema.
• Parquet/Feather/ORC write scaffold dedup via shared _stream_columnar_write helper.

📊 Measured throughput (Windows 10, Python 3.13, 500k-row synthetic)

See docs/benchmark_results_v1.1.3.md for the full per-test trace including external comparison vs Polars / pandas / pyarrow / dlt. Headlines:

• JSON parse: 1,678k rows/sec (orjson-dominated, ~4× write rate)
• NDJSON: 434k write / 543k parse
• Parquet: 278k write / 237k parse
• ORC: 333k write / 239k parse
• Feather: 242k write / 236k parse
• SQLite: 174k write / 218k parse
• CSV / TSV / PSV: ~119k write / ~172k parse

627 tests pass on every PR + push. mypy --strict clean on 55 source files.

📦 Install

pip install incorporator==1.1.3
pip install incorporator[orchestrate]==1.1.3   # +typer for the CLI
pip install incorporator[all]==1.1.3           # everything except [parquet]
pip install incorporator[parquet]==1.1.3       # +pyarrow for Parquet/Feather/ORC

Full changelog: CHANGELOG.md.