Bank statement PDFs are a zoo: what broke our parser

A PDF has no idea it contains a table. It knows glyphs and where on the page to put them. Everything else — that these characters are a column of amounts, that this row wrapped onto the next line, that the number in the far right is a running balance — is inference we layer on top.

We've been building a bank statement parser for a while now. Below are four things that broke it, in the order they taught us something. The last one changed how we think about the whole problem.

1. There is no "the" layout, so we stopped writing one parser

The obvious first design is one parser with branches for the banks you support. It dies quickly. A high-street current account prints a running balance on every line. A credit card prints no balance at all, and its charges increase the amount owed — the opposite sign convention. Nigerian fintech statements print debits and credits in two fixed columns. An "all accounts" statement contains several balance chains stacked in one document.

What actually generalises is structure, not brand. We now run an ordered ladder of layout-generic parsers — credit card, tested per-bank profiles, two-column ledger, debit/credit ledger, multi-section, coordinate-accurate table read, and so on down to a plain-text fallback. Each rung tries, and either produces a result or declines cleanly by returning None.

Adding support for a new bank is usually not a code change. It's discovering the layout family it already belongs to.

2. Every rung has to prove itself against the statement's own arithmetic

A ladder of parsers is only useful if you can tell when a rung is lying. Extraction failures are rarely loud. You very seldom get an exception; you get thirty-eight rows that look completely plausible and are missing the two that mattered.

So each rung is reconcile-gated. A parser's output is only accepted if it agrees with the numbers the statement itself prints — the opening balance, plus every movement we read, landing exactly on the printed closing balance. For a card, that check runs against the summary box instead. Rows that don't reconcile don't get promoted just because they parsed.

This is the single highest-leverage thing in the codebase. The document ships with its own test suite printed on it; you just have to run it.

3. The safety net was eating correct output

Here is the bug that hurt most, because it came from a feature working exactly as designed.

We have a repair pass. If an AI-read statement doesn't reconcile, it walks the running balance and tries progressively more expensive fixes until the chain adds up. Good idea. It rescued plenty of documents.

Then someone converted an American Express statement and the money-in column came back empty. Payments and refunds — gone.

The card parser had done its job correctly, including reconciling against the summary box. Then the repair pass ran, applied a naive opening-to-closing balance check, and found the arithmetic "wrong" — because on a card, purchases increase the balance. The sign convention is inverted. So it dutifully "repaired" a correct parse by dropping the rows that didn't fit its assumption: exactly the credits.

The same trap was waiting for multi-section statements, where each account chains separately and a single opening-to-closing check is meaningless.

The fix is three lines and one idea: a parser that has already proved its own reconciliation must not be second-guessed by a more general repair. If a rung says it reconciled, the repair pass declines. The lesson we took: when you have a component that "fixes" things, be very sure it knows what it is looking at. A repair that can't tell correct from incorrect is just damage with good intentions.

4. Tables with no lines, and prose that parsed as a row

Table extraction libraries mostly key off ruling lines — the actual vector strokes drawn between cells. That works until you meet documents that don't have any.

Plenty of statements are generated by rendering HTML to PDF. Headless Chromium doesn't emit a border as a stroked line; it emits a very thin filled rectangle. To a line-based table finder, a page full of visible borders reports zero lines. Worse, the page's background — one large filled rect enclosing everything — is a perfectly good table candidate if all you check is "rectangular region with content in it." We were confidently extracting a one-by-one table containing the entire page.

Two things fixed it. Choose the best table candidate on the page by scoring rather than taking the first, and derive column boundaries from the widest real row. Then drop the giveaway: a "row" that is a single cell containing more than 200 characters, alone on its line, isn't a row. It's the surrounding prose that the container rectangle swallowed. Real table cells are short. Paragraphs are not.

What we'd tell anyone starting this

• Find the arithmetic the document already contains,and check against it. Statements have balance chains; invoices have line items that sum to a subtotal, plus tax, making a total. If your domain has this property, it is worth more than any model upgrade.
• Make every component able to decline. Returning "I'm not confident" needs to be as easy as returning an answer, or components will answer anyway.
• Be suspicious of your repair logic. Anything that silently rewrites output can silently rewrite correct output.
• Fail early and honestly. A PDF with no text layer is a scan. We detect that up front and say so, rather than returning the handful of garbage rows an OCR-less read would produce. Users forgive "we can't do this one." They don't forgive a spreadsheet that's subtly wrong, because they find out downstream — in their books, in a tax return, in front of a client.

That last point is the whole thesis, really. In document extraction the expensive failure isn't the file you refuse. It's the file you got 97% right without saying which 3% was missing.

The parser is the engine behind StatementDecoder, which converts statements, invoices and receipts and marks each conversion as reconciled — or tells you plainly when the numbers don't add up. If you have a bank PDF that breaks it, we genuinely want to see it; those are what the ladder gets built from. Related reading: why scanned statements are a different problem and the anatomy of a statement descriptor.