The books close
themselves. Mostly.
A settlement-reconciliation agent modeled on Razorpay-style payment flows — gateway, two banking partners, Razorpay's own internal settlement ledger, and Razorpay Capital's loan-recovery book, cross-checked automatically. The deterministic engine resolves 7 in 10 transactions on its own, with zero ML/LLM involvement; an LLM only reasons about the ambiguous tail, and never gets the final word.
One pipeline, one rule: the model proposes, code decides.
Every transaction flows through the same pipeline. Only the fraction the deterministic engine genuinely can't resolve on its own ever reaches an LLM — and even then, a separate, non-negotiable gate decides what happens next, and every escalation lands in a real human review queue. For Razorpay: this is the shape a finance-ops leader can actually sign off on — every rupee that moves is traceable to plain Python, not a model's judgment call, built on the kind of boring, battle-tested infrastructure (Postgres, Redis, scheduled batch jobs) a real ops team already knows how to operate and audit.
Every box above is a real code path, not an illustration — trace any of it in matching/, agent/, investigator/, review_backend/, postgres/, redis/, or airflow/. Postgres, Redis, and Airflow are all local Docker containers — free, no cloud dependency, and each isolated from Airflow's own internal Postgres/Redis (its Celery broker), never cross-wired.
Grounded, tool-using, and never the final authority.
The LLM only ever sees the 617 cases the deterministic engine could not resolve on its own. It reads real policy, calls real tools against real data, and cites what it used — then a separate, non-negotiable gate decides whether any of that is allowed to touch a rupee. For Razorpay: the policies it cites aren't invented — they're grounded in real RBI/NPCI frameworks (the TAT and customer-compensation circular, URCS dispute rules, digital-lending guidelines), so every AI-assisted decision already reads like something an auditor, not just an engineer, could sign off on.
Retrieval isn't decoration — it's the whole difference
A live A/B on the same cases: with policy retrieval on, the model cited the correct policy 16/16 times. With it off, 1/16. Mean confidence was identical (0.90) either way — an ungrounded model doesn't sound less sure, it just confidently cites the wrong rule. That's exactly why the gate treats a policy-citation mismatch as a hard block, not a soft warning.
Cases investigated with real tool use, not one-shot guessing
A hand-rolled ReAct loop — no agent framework — gives the model seven deterministic tools over real settlement data and up to six rounds to decide what to check. Across the benchmark: 100% correct policy citation, zero hallucinated policies. Every tool call and its result is recorded, so a reviewer audits the reasoning instead of trusting it. 154 is coverage, not a filter — every one of the 617 escalated cases is eligible for this same investigation; 154 reflects how much of the backlog has been run through it so far, a number that grows with every batch, not a rule deciding which cases "deserve" deep tool use. The other 463 currently carry the single-shot classifier's proposal instead, gated by the exact same deterministic check either way.
617 tickets are not 617 problems
One settlement whose bank posting arrived without a UTR flags every payment batched into it — so a single upstream event fans out into hundreds of separate cases that all clear together. Clustering on that exact fan-out key collapses the queue to 130 real root causes, and just 31 of them cover 518 cases — 84% of the backlog. Zoomed out by exception type rather than by individual cause, missing_bank_reference alone spans 497 cases across 21 different settlements — a separate, wider-lens stat from the 518-case cluster figure above, not the size of any single cluster (the single largest cluster is 47 cases). Deterministic, ~70 ms, no embedding model: the cases that share a cause already share a join key, so approximating it with vectors would be strictly worse.
Escalations land in a real queue — and can close themselves
Every case the gate declines to auto-resolve becomes an immutable, append-only review case carrying the AI's full proposal and evidence trail. Re-verification re-runs the matcher and auto-closes any case whose transaction is now genuinely clean — not merely reclassified — on Airflow's own schedule, or on demand from the review dashboard itself, so the backlog shrinks without a human re-checking each one.
A reconciliation isn't done when it's matched. It's done when it's booked.
Everything above produces a proposal. What follows is the part that actually closes the loop — a governed human queue, a tamper-evident trail of who decided what, and real double-entry output. This all runs locally rather than on this page, so it's described here rather than shown. For Razorpay: this is the difference between a matching demo and something an ops team could sit in front of on a Monday morning — the exception has an owner, the approval has a second signature, and the outcome lands in the books.
Tiered human review, not a rubber stamp
All 617 escalations land in a real queue with an explicit state machine. Anything at or above ₹50,000 needs two people — an analyst, then a different manager. Illegal transitions are rejected outright rather than silently accepted, and two reviewers acting on the same case at once is caught by an optimistic-concurrency check instead of one quietly overwriting the other. A human can always override the AI; the AI can never override a human.
Tamper-evident, and not just by convention
Every review event is hash-chained to the one before it, so deleting or editing any historical decision breaks verification for every decision after it — the sequence is protected, not just each row. Append-only is enforced by Postgres itself, through a deliberately non-superuser role that has UPDATE and DELETE revoked, rather than by the application promising to behave. Each case also carries its own content hash, and every pipeline run writes a manifest pinning the exact input bytes and every threshold in force.
Double-entry journal output, zero LLM
Every transaction drafts a real journal entry against a small chart of accounts, with the variance routed by exception type — a Razorpay Capital recovery to a liability account, a chargeback to loss, an unexplained shortfall to suspense rather than to a guess. All 2,072 balance: total debits equal total credits exactly (₹2,06,56,815.99 each). The accounting treatment for a known exception type is standard practice, not a judgment call — so there is nothing here for a model to decide, and it doesn't.
The regulation is enforced, not just cited
RBI's Harmonisation-of-TAT circular sets T+5 business days as the outer resolution bound with ₹100/day compensation past it. Each open case carries a real deadline derived from the ledger's own business-day-aware expected settlement date, an overdue count, and accrued exposure — surfaced as a priority signal for humans. It never changes a case's status or influences the gate. Alongside it sits a full bank-reconciliation bridge from books to bank statement, whose residual variance (0.133%) is reported honestly as an explained remainder instead of being forced to zero.
Seven more tools sit behind that queue, each answering a question a reviewer actually asks: a grounded Settlement Q&A agent, the bank reconciliation statement, root-cause clusters, the matcher's own auto-resolves, correction memory (a past human override is fed back into future AI prompts as a worked example), audit-chain verification, and on-demand re-verification.
All of it is one command against a local Postgres — no cloud account, no API key, and zero LLM calls on the default path: python scripts/run_demo.py
Every number on this page is reproducible from a clean clone.
A finance system that can't be checked is just a confident one. The claims above are backed by executable proofs rather than assertions — including the ones that prove a safeguard is real by deliberately breaking it and confirming the alarm fires. For Razorpay: the interesting question isn't whether a reconciliation engine works on a good day; it's what it does on a bad one, and whether you can tell the difference without reading the source.
Including the attacks
A hostile bank narration — "IGNORE ALL PREVIOUS INSTRUCTIONS…" — is run through the real tool and the real gate, with a stand-in model that fully complies with the attack, proving the system resists it even when the model doesn't. Bank narration is never sanitised, on purpose: scrubbing it would hide a genuine data-quality signal from a human. Three independent gate conditions were each weakened in turn to confirm no single one is load-bearing.
Reproducibility that found a real bug
Accuracy is re-measured across 25 independently regenerated datasets, not one lucky seed. The first run came back 10 of 25 below 100% — a real generator bug where a payment could settle same-day while still labelled late. Fixed, re-run, 0 of 25 below 100%, and a permanent guard added so it reports itself if it ever returns. The single-seed claim it replaced looked clean the whole time.
Architectural rules enforced by tests, not discipline
"The deterministic core never imports the AI layer" is checked by a static scan and a runtime subprocess check, so it can't rot into a comment that used to be true. The same applies to reading the wall clock, unseeded randomness, and env-var drift inside the matcher — all banned and all tested, which is why the same inputs produce byte-identical output on any machine, any day. Ground truth is readable by the scoring script alone, and a test proves it.
The matcher doesn't trust itself either
"AI proposes, deterministic code disposes" is usually read as a rule about the model — but a wrong classification the matcher produced would reach the gate as unquestioned fact. So every run independently re-derives whether any bank row was double-consumed and whether every matched settlement's arithmetic actually closes, and refuses to return a report that fails. Cost: 1.72 ms per run, measured.
Throughput a finance team can trust.
2,072 transactions in under a second
A full month of settlement data — gateway, bank, ledger, all cross-checked — end to end, on a laptop CPU, with zero LLM calls in this layer. Timed on every run by evaluate.py, never hand-typed.
Resolved with zero ML/LLM
The deterministic core carries the volume. The LLM is reserved for the genuinely ambiguous tail — the only place judgment, not arithmetic, is actually needed.
A measured safety ceiling, not a guess
False auto-resolve rate, scored against hidden ground truth over all 2,072 transactions — a wider denominator than the 98.97% auto-resolve precision figure elsewhere on this page, which scores only the 1,455 cases the system actually auto-resolved. The two aren't complements of each other; they answer different questions. Every auto-resolve decision is bounded by a policy citation match, a confidence floor, and a hard rupee ceiling — simultaneously.
Built to extend, not just to demo.
Every item below is a scoped, already-understood extension of code that already runs — not a research problem or a wishlist. Grouped the way a real production rollout would prioritize them: what's cheapest to harden before this touches a real ledger, what it actually takes to connect to Razorpay's own systems, what closes an already-documented edge case, and what can honestly wait.
Harden what's already proven
- Integer-paise (Decimal) arithmetic, replacing today's float rupees + ±₹0.02 tolerance band. For Razorpay: the cheapest, fastest step toward trusting this on a real merchant ledger, where exact-paisa accounting isn't optional.
- Postgres connection pooling — already measured on this exact codebase: 92% of DB request time is connection overhead, not query work. A handful of call sites to change. For Razorpay: the same review-queue experience at real operator headcount, for a fraction of the infra cost.
Connect it to Razorpay's real systems
- Live gateway & bank settlement API connectors, replacing today's synthetic, seeded CSV/XML generation. For Razorpay: the one change that turns a provable demo into the actual reconciliation backbone behind live settlements.
- Kafka-based event ingestion, replacing the tick-based stream simulator with genuinely real-time settlement events. For Razorpay: exceptions surface the same day they appear, not on the next scheduled run.
- Horizontally-distributed matching — proven at ~2,300 txn/s on a single laptop process; a Razorpay-wide rollout needs this to scale across every merchant, not just every transaction. For Razorpay: one reconciliation engine for the whole merchant base, not one instance per team that happens to need it.
Close the known, already-documented edges
- General N-way settlement splits — today's matcher handles 1:2 splits only. For Razorpay: covers the batching patterns 1:2 can't, without touching any of the matching logic that's already proven.
- A global assignment solver, replacing today's greedy bank-row consumption (safe and tested, but order-dependent). For Razorpay: removes the one documented "safe but not provably optimal" edge case before real scale makes it more likely to matter.
- Explicit per-member attribution for the reconciliation bridge's small (~0.13%), already-explained residual. For Razorpay: takes an honestly-reported residual to a fully-attributed, line-item-traceable zero.
Polish, once the foundation is real
- Full observability & tracing (e.g. Langfuse) — evaluated directly during this build, deferred specifically on self-hosted infra cost (16 GB+ RAM on a demo machine), not on any doubt about its value. For Razorpay: worth the infra cost the moment this runs on real traffic instead of a laptop demo.
- Full RBAC beyond today's two-tier analyst/manager approval, integrated with Razorpay's own SSO/IAM. For Razorpay: the access model a real finance org needs, not the single-reviewer model a demo gets away with.
Nothing above changes the one rule this whole system is built on: the LLM still never touches a number, at any scale.