NameetP/pdfmux

Two defects that had the same shape: a signal was computed and then thrown away. One of them is fixed. The other is not — see the retraction below.

Fixed — Arabic documents were never routed to an Arabic-capable backend

_classify_to_page_type has always returned "arabic" for them. ROUTING_MATRIX had no "arabic" rows, so every Arabic document fell through to DEFAULT_CHAIN — whose BALANCED arm is ("opendataloader", "pymupdf") and never reaches an LLM.

That made the README's documented behaviour untrue, and contradicted the code's own note that "PyMuPDF/RapidOCR are unsuitable on Arabic-heavy docs."

Added the missing rows. ECONOMY stays free (pymupdf; BiDi is applied post-extraction as before). BALANCED and PREMIUM lead with llm, which resolves to the best available provider, and fall through to pymupdf when none is configured, so nothing breaks without an API key.

Added — table_truncated flag, and a retraction of the number first published with it

The flag is real: _has_table tests presence, so a table cut from 40 rows to 3 still passes. table_truncated compares row cardinality, document-wide and for the largest contiguous table block.

⚠️ The "7 of 7" improvement figure originally published in these notes was measured incorrectly and is retracted.

It applied the seeding rule to the hand-written ground-truth Markdown and tested the flag directly. But verify_extraction compares an extraction against the source re-derived from the PDF — and that re-derived text typically contains no Markdown pipe-table markup at all. Measured: 0 pipe rows for both bls-empsit and irish-census-1926, against 11 and 7 in their hand-written ground truth. With zero source rows the >= 4 guard never passes, so the check cannot fire.

The honest number, measured end-to-end with validate_verifier.py on this release:

| | |
|---|---|
| Applicable seeded truncations | 4 |
| Detected | 1 |
| False-negative rate | 75% (against 80% before the flag existed) |
| Still missed | arxiv-2203.02155-instructgpt, bls-empsit, irish-census-1926 |

Treat table completeness as unsolved. The flag helps only when the source is itself Markdown (--extracted Markdown-vs-Markdown), not when the source is a PDF — which is the common case.

Run over 23 of 24 corpus documents; gao-24-106214 could not be fetched (gao.gov returns 403 to automated fetch regardless of User-Agent, no Wayback capture). bls-empsit was recovered from a Wayback snapshot whose sha256 matches the pin exactly.

Still purely additive. _has_table, table_integrity and all five pre-registered thresholds are byte-identical, so the published false-positive/false-negative figures remain valid for the signals they measured. A flagged page surfaces as review, never as a new hard failure.

759 tests pass.