The AI System That Never Touches a Live Transaction
How Sphere separates probabilistic AI research from deterministic tax execution
TL;DR: Sphere’s Tax Review and Assessment Model (TRAM) is the clearest public example of where probabilistic AI belongs in a financial system: on the research side, never on the transaction side. The AI monitors tax authorities, retrieves legal evidence, and proposes determinations with citations. Tax experts approve or correct every proposal before it becomes a versioned rule in a deterministic engine, and only that engine touches live transactions. The system decomposes into the six layers every production AI system shares: knowledge acquisition, ingestion, retrieval, reasoning, review, and execution. TRAM’s implementation of each layer is a working answer to a generic AI system design problem, from change-detecting crawlers to a review gate that manufactures training data to a fully deterministic serving path. Vendor-published results: no-edit accuracy rose from under 65 percent to over 90 percent while median expert review fell from two minutes to under ten seconds. Steal the boundary before you steal any component.
Sphere’s TRAM shows how to run AI safely in a financial system: put the model where an error creates review work instead of customer liability. A probabilistic research plane proposes tax determinations with citations, one expert gate promotes them into versioned rules, and a deterministic engine calculates tax on live transactions. Nothing generative runs at checkout.
TRAM (Tax Review and Assessment Model) is Sphere’s AI research system for global indirect tax. It ingests statutes, regulations, guidance, and rulings across jurisdictions, retrieves controlling authority for each product and region, and proposes taxability determinations with citations. Experts review every proposal before it becomes a versioned rule in a deterministic tax engine. The design generalizes to any vertical where a wrong answer is a legal liability.
In this AI System Design:
Why “correct most of the time” fails in tax, and the reliability bar auditors actually set
The three-plane architecture: probabilistic research, a human promotion gate, a deterministic runtime
The six layers of AI system design: knowledge acquisition, ingestion, retrieval, reasoning, review, and execution, each taught through TRAM’s implementation
Harness lessons made concrete: replaceable models behind a stable proposal contract
The four operational metrics that moved when benchmark scores would have stayed flat, and the questions to run against your own system
The reliability bar: why tax needs an AI research system
Starting with the customer, Sphere sells tax compliance to companies such as Lovable, Replit, Windsurf, Deel, and ElevenLabs, businesses that sell software into dozens of countries almost immediately. A Stripe study cited in Sphere’s Series A announcement found most startups sell into 90 or more countries by the end of their second year. Each jurisdiction taxes each product category differently, changes rules on its own schedule, and holds the seller liable for errors.
Tax calculation looks deterministic from the outside. A transaction enters an API; the system resolves the product, location, exemptions, and rate; it returns an amount. The difficult part is producing the rule behind that calculation. Tax knowledge lives in statutes, regulations, administrative bulletins, court decisions, and government PDFs that change without warning. An authority may publish an interpretation today that takes effect in three months, or quietly replace a document while keeping the same URL.
The bar has two parts. The calculated tax must be correct for that product, in that jurisdiction, on that date. And the reasoning must trace to controlling authority, because tax authorities audit conclusions against statutes. An answer without a citation fails even when it happens to be right.
Incumbent tax engines meet this bar with large content teams manually reading legislation, a process Sphere’s engineering write-up describes as labor intensive and error prone, and the main obstacle to covering new jurisdictions and product types. TRAM automates the research while keeping the bar where auditors put it.
The architecture: two workloads, three planes
The workload splits cleanly in two. Research needs exploration: searching a changing corpus, interpreting legal language, weighing conflicting evidence, expressing uncertainty. Probabilistic AI is good at this, and a research error costs one review task. Transactions need certainty: identical inputs, identical outputs, low latency, complete audit trail. Deterministic software is good at this, and a transaction error costs real money and regulatory exposure.
Forcing both workloads into one agent produces an unstable system. TRAM refuses to, and the result is three planes:
Sphere states that every determination and taxonomy proposal is reviewed before being pushed into its production tax engine. Model output never becomes production truth because it looks convincing. It becomes a candidate; a human-approved, versioned rule becomes production truth.
The best mental model is an evidence-to-rule compiler. A compiler transforms one representation into another through controlled stages, and TRAM transforms unstructured legal material into executable rules: publication, versioned sections, evidence package, AI proposal, expert-approved decision, machine-readable rule, deterministic execution. That frame changes the design goal. The output cannot be a well-written answer. It must be structured, reviewable, versioned, schedulable, and reversible, an artifact that can safely cross into production.
The model proposes. The expert promotes. The runtime executes.
The six layers of an AI system
Strip the tax domain away and TRAM decomposes into six layers that every production AI system contains, whether the team has named them or fallen into them: knowledge acquisition (how the system learns its ground truth changed), ingestion (what the unit of retrieval is), retrieval (how evidence gets assembled), reasoning (how model output becomes a reviewable artifact), review (where output gains production authority), and execution (what actually serves the user).
Layer 1: Knowledge acquisition. Monitor the source of truth
Every AI system grounded in external truth faces the same design question: how does the system find out that its ground truth changed? For TRAM the ground truth is law, and the first problem is knowing when it moves. Sphere built WARP (Web Automation Reimagined Purposefully), an in-house crawler that tracks where each authority publishes, schedules recurring crawls, detects new or changed documents, and triggers re-ingestion.
The hard part is deciding whether a change matters. Government sites constantly touch navigation, banners, and templates; a naive content hash alarms on all of it. The system needs to detect changes in legal meaning, and when it finds one, emit something an operations team can act on:
A dependency graph between sources, evidence, and production rules lets the system answer the question that matters operationally: which active conclusions might now be wrong? When a monitored source changes, TRAM re-reviews the determinations that cited it and flags updates for expert approval. Retrieval over a static corpus answers yesterday’s law; monitoring turns the corpus into a live dependency.
Layer 2: Ingestion. Chunking is schema design
The ingestion layer decides the unit of retrieval, and that decision is schema design in any domain: the way information is divided determines what the system can later understand. Sphere identifies document splitting as a major contributor to accuracy, which deserves attention because chunking is the stage most teams treat as a solved default.
Fixed-size chunking is dangerous for legal text. A statute’s general rule, its exception, and its effective date often sit in adjacent blocks. Split them apart and retrieval returns “digitally delivered software is taxable” while the controlling exemption for remotely accessed enterprise software sits in another chunk. The sentence retrieved is accurate; the decision built on it is wrong.
Legal documents carry a natural hierarchy: title, chapter, section, subsection, paragraph. TRAM preserves it with two splitters. A rules-based splitter encodes the heading and numbering patterns common in tax materials. For messy documents that fit no pattern, an LLM writes bespoke, document-specific splitting rules that then execute in code. The generative step produces inspectable, reusable logic instead of opaque one-off output, which keeps the pipeline debuggable when a jurisdiction publishes something strange.
Each stored section keeps its jurisdiction, authority type, effective dates, and parent chain, and non-English documents carry an English rendering alongside the authoritative original for consistent review.
In high-stakes RAG systems, chunking is part of the domain model.
Layer 3: Retrieval. Hybrid search in a bounded loop
The retrieval layer answers one question: how does the system assemble the smallest evidence set that supports a defensible output? Vocabulary mismatch is the first obstacle. A product team says “cloud-hosted application used by businesses.” The statute says “remotely accessed enterprise software.” Dense retrieval bridges that wording gap. But legal meaning also rides on exact phrases, defined terms, and statute numbers, and dense retrieval fuzzes past them, returning plausible neighbors instead of controlling authority. Sphere reports that dense and sparse retrieval together consistently outperform either alone, with sections found by both methods boosted.
The pattern holds outside tax. Anthropic’s contextual retrieval work pairs embeddings with BM25 for the same reason and measured a 67 percent reduction in failed retrievals when hybrid retrieval combines with reranking.
One search pass is still too shallow, because the section matching a query rarely contains every definition and exception needed to interpret it. TRAM runs retrieval as a bounded loop: retrieve candidates from both indexes, rerank, drop the weakest, expand survivors by pulling neighboring sections through the parent chain, and repeat until the evidence fits the context budget. The goal is the smallest evidence package that supports a defensible decision, with the citation trail falling out as a structural byproduct.
Layer 4: Reasoning. A structured proposal behind a stable contract
The reasoning layer is where most AI system designs quietly couple themselves to one model vendor. The countermeasure is an output contract: the model returns a schema, never an essay.
The schema is what makes everything downstream possible. Code validates it. A verifier can check each claim against its cited sections, because a citation can exist and still fail to support the claim; citation presence and citation entailment are different tests. Reviewers edit one field instead of rewriting an answer.
And the model becomes replaceable, which stopped being theoretical this year. Sphere’s documented implementation fine-tunes OpenAI reasoning models through reinforcement fine-tuning on its own expert feedback.
Layer 5: Review. The promotion gate that manufactures training data
Every high-stakes AI system needs a promotion boundary, the point where probabilistic output gains production authority. In TRAM that point is exactly one place: expert review. On the TWIML AI Podcast, Sphere’s head of engineering frames the workflow as legal review rather than labeling: experts practice their profession, and the system captures the judgment.
The capture is the clever part. When a reviewer modifies or rejects a proposal, the portal requires a short explanation, stored with the task, the retrieved evidence, and the approved outcome:
Every correction becomes a retrieval fix, a fine-tuning example, or a regression test; Sphere says TRAM was trained on thousands of hours of feedback from expert researchers, generated as a byproduct of work the experts had to do anyway. Reviewers also flag determinations into an eval set that runs on every model or pipeline change, the regression gate that separates improving a system from merely changing it. A review tool that records only approve or reject throws away the judgment the company already paid for.
Approved determinations become versioned rules with two dates, the approval date and the legal effective date, because an expert can approve a change today and schedule it for the correct date. Hard invariants live in code rather than prompts: an unapproved, expired, or superseded rule can never activate.
Layer 6: Execution. A deterministic runtime with audit lineage
The execution layer serves the user, and its design rule is subtraction: remove everything probabilistic from the serving path. In TRAM, a customer transaction resolves the product and jurisdiction, selects the active rule version for the transaction date, executes fixed logic, and logs the rule version used. No crawling, no vector search, no model call, no waiting on an expert. The runtime becomes an ordinary software reliability problem: immutable rule artifacts, replay tests, canaries, rollback.
Every result stays traceable backward. That lineage graph is the audit artifact. A generated explanation tells a story about a decision; the graph proves which rule ran, who approved it, and which version of the law supported it, and every edge stays queryable after later versions publish.
AI and deterministic software can belong to the same product without belonging to the same execution path.
Does long context make this obsolete?
The strongest case against the pipeline is the context window. Anthropic’s own guidance says a corpus under roughly 200,000 tokens can simply be placed in the prompt, skipping retrieval, and windows keep growing. If a model can read everything, the machinery above looks like legacy plumbing.
The argument fails on what the system actually runs on. A monitored, multilingual legal corpus across 100-plus regions sits orders of magnitude beyond any window and grows daily. Stuffing documents into a prompt creates no versioning, no effective-date filtering, no dependency tracking, no promotion control, no audit lineage, no historical replay. And at continuous determination volume, full-corpus prompting prices itself out.
The hard problem was never fitting documents into a model. The hard problem is managing authority over time. Bigger windows change where the retrieval budget goes; they have not changed whether the audit trail needs one.
FDE Decision Log
Four key decisions shaped the system:
Build custom monitoring: Sphere built WARP because detecting meaningful legal changes is central to keeping rules current.
Preserve document structure: It avoided basic fixed-size chunking because legal meaning depends on sections, exceptions, and effective dates staying together.
Own the feedback data: It used expert corrections to improve the models, while keeping the proposal format independent of any one model provider.
Keep production deterministic: Live transactions use approved rules, not model output, making results predictable, traceable, and auditable.
Production Failure Modes
Most failures are silent.
A crawler can miss an update. Retrieval can separate a rule from its exception. A citation can look valid without supporting the claim. Reviewers can approve too quickly. A future rule can activate early. One jurisdiction can regress while overall scores improve.
The system may still return polished results even as the evidence becomes less reliable. That is why observability must track meaning and correctness, not just uptime and errors.
Operational Metrics
Four metrics matter:
No-edit rate: How often experts approve a proposal without changes. Sphere reports over 90%.
Review time: How long expert approval takes. The median is under 10 seconds.
Change-to-rule time: How quickly a legal change becomes an approved, correctly scheduled rule. Crawl frequency alone does not measure freshness.
Replay consistency: Historical transactions must produce the same result when rerun with the same rule version.
Track each metric by jurisdiction, product, and language. A strong overall score can still hide serious failures in a smaller segment.
Lessons from the Field
Three lessons apply to almost any regulated industry.
First, use AI where mistakes are easier to catch and fix. Let the model help with research and recommendations, but place a review step before anything reaches production. After approval, use normal software rules to carry out the decision. Where you place the model matters more than which model you choose.
Second, make the review process useful. When an expert corrects the AI, save what changed and why. Those corrections can improve the system, become training data, and turn into future test cases.
Third, treat your source material as something that keeps changing. Monitor it, keep old versions, and track when each rule becomes active. In regulated industries, the “right answer” can change over time.
This approach works in many areas, including insurance, healthcare, compliance, security, and contracts.
To review your own system, trace every path that AI output can take into production.
For each path, ask:
What stops a bad output before it goes live?
What metric would tell us the system is getting worse?
Any path without both a clear gate and a clear metric should be your next design review.