Build Log #6: Nothing Gets Deleted on a Guess

With one real parser working, the next problem showed up immediately: I re-upload a statement, or two statements overlap by a few weeks, and now the same transaction exists twice. This entry is deduplication and the analytics engine that depends on it — the first thing in the project that actually reads the parsed data instead of producing it.

Two passes, in order

Statement-level dedup runs first: if an account, period, and both balances match an existing statement exactly, it's the same PDF re-uploaded, and the whole statement plus every transaction in it gets marked a duplicate. Only statements that survive that check go to transaction-level dedup, which looks for overlapping periods on the same account — the case where a "last 90 days" statement shares three weeks with an already-imported monthly one.

A clean one-to-one match inside that overlap collapses automatically. Anything with ambiguous cardinality — say, three identical $12 charges to the same merchant on the same day, split across two overlapping statements — doesn't get resolved by a guess. Every row stays, the newer ones get flagged as a possible duplicate, and a person decides.

A missed duplicate costs a slightly inflated total until someone notices. A wrongly deleted transaction costs the one thing this app is supposed to provide: a number you can trust.

Nothing is ever deleted by this system. A duplicate row just stops counting toward the ledger every later calculation reads from — which means undoing a mistake is always possible, because the evidence is still sitting right there in the database.

The analytics engine reads a rule, not a table

There's no dedicated "ledger" table. It's a rule — a transaction counts if its own dedup status isn't DUPLICATE and its statement's isn't either — applied fresh every time analytics runs. Cash flow, spending by category, recurring charges, merchant totals, and month-over-month trends are all deterministic Python over that rule. No LLM sits anywhere near a number.

Recurring-charge detection has to tolerate a subscription raising its price without losing the pattern: a merchant qualifies once it has at least three charges whose dates land close to a known cadence — weekly, biweekly, monthly, or annual — and whose amounts stay within about 15% of the group's typical value. A charge that goes from $15.99 to $16.49 is still the same subscription, still monthly, still recurring.

Next

Categorization, and the Privacy Gateway that has to exist before any of this data is allowed near a model.