All notes

Sep 2026 · 5 min read

Published

Build Log #7: The Gateway That Only Lets Four Fields Through

AuthorKervintz Noel
Filed underAI Engineering · Claude Code · Building in Public · Python
SeriesPart 7 of 11 · View the full case study

This is the post I was most careful about, because it's the one where a mistake doesn't show up as a wrong dollar figure — it shows up as somebody's account number or a Zelle counterparty's name leaving the machine.

Deterministic first, always

A transaction's category resolves through a fixed order: a user's own override, then a saved merchant rule ("always categorize Uber as Transportation"), then a deterministic prediction, and only if none of those apply does the transaction fall to Uncategorized and land in Review. The automated guess is stored separately from any override or rule, which sounds like a small detail until you delete a rule and everything that rule used to touch snaps back to its prediction instead of needing a bulk rewrite.

The deterministic engine — a fixed merchant-alias map and a keyword scan — lives in code, not database rows. Every time that list changes it's a code edit and a test, not a data migration silently drifting out of sync with what's actually deployed.

Only a transaction the deterministic pass can't place gets an LLM call, one per unique merchant rather than one per transaction. A confident deterministic hit and a confident LLM hit are treated the same way: below a 0.75 confidence gate, both go to Review instead of showing a guess as if it were certain.

The allowlist, not a scrubber

The thing I didn't want to build was a function that tries to strip sensitive fields from a real transaction before sending it out — because "tries to strip" means it can also fail to. Instead, the payload that leaves the machine is its own type: exactly four fields — merchant, description, amount, direction — built from primitives, not assembled by trimming a real transaction object down. There is no fifth field for a bug to accidentally forget to remove, because the type itself doesn't have room for one.

  1. 01Free text is separately sanitized: SSNs, emails, spaced-out card numbers, phone numbers, and long digit runs are stripped
  2. 02A Zelle, Venmo, Cash App, or PayPal transfer line has the entire counterparty tail dropped, not just obscured
  3. 03A "WIRE TO <name>" or "TRANSFER FROM <name>" line loses the name the same way

And if sanitization can't guarantee the result is clean, the gateway fails closed — no LLM call happens, and the transaction goes to Review exactly like a low-confidence prediction would. Silence is the safe failure mode here, not a best-effort guess.

The privacy boundary isn't "we try to remove the sensitive fields." It's a payload type that cannot physically hold a fifth field.

Provider choice, made honestly

OpenAI is the primary provider, Anthropic is the fallback — but the fallback only fires when the primary actually fails: a timeout, a rate limit, a broken response. It never fires just because the primary's answer was weak. A weak answer goes to Review like any other, on its own merits, without a second provider getting a chance to overrule it.

The test I trust most here isn't a coverage percentage. It's one that takes a transaction description containing a real account number, a name, a phone number, and an email, sends it through the whole pipeline, and asserts that the payload actually recorded on the wire has none of them — and exactly the four allowed fields, nothing else.

With no provider key configured at all, categorization behaves byte-for-byte the same as if the LLM layer didn't exist. The app was never allowed to depend on it.


Next

The frontend. Everything above this line had been verified with curl and pytest — nothing on a screen yet.

I write these as I go. You can follow along here or on Hashnode, where I'll start cross-posting.

All notes