OCR-AI

A point-of-sale provider in Saudi Arabia receives receipts and invoices from a large fleet of devices — in Arabic and English, photographed at angles, scanned badly, sometimes running to many pages. Saudi tax rules (ZATCA) require specific fields in a specific shape, so the output cannot be approximately right: a wrong VAT number or total is a compliance problem, not a formatting one. Conventional OCR lifts characters off the page but does not know which characters are a line item and which are a total. A single model asked to do the whole job returns an answer with equal confidence whether it read the total cleanly or guessed at it, which gives the consuming system nothing to act on.

A production LLM document-intelligence platform. It takes a receipt or invoice image, runs it through a LangGraph agent workflow, and returns a validated 72-field structured record shaped for ZATCA tax compliance — with a confidence value per field rather than one score for the whole document, so a consumer can accept the fields that are solid and route only the uncertain ones for review. The same codebase ships as one Docker image with four entrypoints: a Gradio UI, a FastAPI webhook service, a Kafka consumer, and a batch CLI. It is 30 modules across four deployable services.

Wateer's engineering team consumes it as a service inside a cross-team Kafka event pipeline: receipts captured at point-of-sale devices arrive as events and come back as structured records. Operators use the Gradio UI to run a single document by hand and inspect what the ensemble produced and how confident each field is. The batch CLI is for bulk runs over existing document sets rather than live traffic.

• Arabic and English extraction

  Receipts and invoices are parsed in either language, including mixed-language documents, without a separate pipeline per language.

• ZATCA-shaped schema

  Output conforms to a 72-field type-constrained schema built around what Saudi tax compliance requires, so the record is usable as-is rather than needing a mapping step.

• Per-field confidence

  Confidence is reported for each field, not for the document as a whole. A record with a certain total and an uncertain VAT number says exactly that.

• Two extraction workflows

  A 5-node self-correcting single-model workflow for the cheaper path, and a 10-node multi-model ensemble for documents where agreement between models is worth paying for.

• Cross-model agreement scoring

  Where the ensemble runs, a deterministic scorer measures how far the models agreed on each field and derives the confidence from that, without asking a model to rate itself.

• Long-document handling

  Multi-page invoices are chunked against a token budget and reduced back into a single record, so document length is not a hard limit.

• Bulk batch runner

  A command-line runner processes large document sets with bounded concurrency, so a backfill does not exhaust rate limits or memory.

• Signed webhook intake

  Inbound webhooks are authenticated with runtime-selectable strategies, including HMAC-SHA256 signature verification, and are rejected when configuration is missing or wrong.

• Accuracy evaluation harness

  A weighted evaluation harness scores extraction accuracy across a document set, weighting fields by how much getting them wrong actually costs.

Two LangGraph workflows

A 5-node self-correcting single-model graph and a 10-node multi-model ensemble graph. Extraction is expressed as an explicit graph of steps with defined transitions rather than as a chain of prompt calls, which makes the retry and arbitration paths inspectable.

Typed schema as the contract

A 72-field Pydantic schema with a lenient shape and strict semantics: a field may be absent, but a field that is present must satisfy its type constraint. Validation is the boundary between the model's output and everything downstream.

Judge and scorer, deliberately separate

An LLM-as-judge arbiter reconciles candidate values from the ensemble. A separate deterministic, model-independent scorer computes per-field confidence from cross-model agreement. The scorer sits outside the LLM boundary by design.

Kafka integration

An aiokafka consumer joined to a cross-team event pipeline, keyed by device so all events from one point-of-sale device land on the same partition and stay ordered, against a documented and versioned event contract.

Pluggable webhook authentication

Auth strategies are selectable at runtime behind one interface, with HMAC-SHA256 verification using constant-time comparison and fail-closed behaviour on misconfiguration. Eight end-to-end tests cover the failure modes, not just the success path.

Context and cost engineering

A token-budget-aware map-reduce chunker splits long documents to a budget, extracts per chunk, and reduces to one record; a bounded-concurrency runner caps how much work is in flight. Context window and spend are treated as operating constraints of the design.

One image, four entrypoints

Gradio UI, FastAPI webhook, Kafka consumer and batch CLI all ship from a single Docker image, with health checks and restart supervision, so the four deployment modes cannot drift apart in behaviour.

1. 01

   The hardest question is not extraction, it is who grades the extraction. If a model rates its own confidence — or a sibling model rates it — the score reflects the model's self-belief, and models are most confidently wrong on exactly the inputs that matter: a smudged total, a rotated scan, an Arabic field in an unusual font.

   What I did

   Scoring was split in two. An LLM-as-judge arbiter reconciles the candidate values the ensemble produced, and a separate deterministic, model-independent scorer computes confidence from how far the models agreed, field by field. Confidence becomes a property of the ensemble's agreement rather than any model's opinion of itself, and because it is per field, the consuming system can accept 68 clean fields and route the four uncertain ones to a person.

2. 02

   Real receipts are partly unreadable. A strict schema rejects the entire document when one field is illegible, which discards the seventy fields that were read correctly — the worst possible trade for a batch of thousands of scans.

   What I did

   The 72-field Pydantic schema is lenient in shape and strict in semantics: fields may be missing, but anything present must satisfy its type constraint. Partial OCR degrades into a partial record rather than an exception, and per-field confidence tells the consumer which parts to re-check. The 5-node self-correcting workflow gives the single-model path a chance to repair output that failed validation instead of returning it broken.

3. 03

   Invoices run to many pages, and a long document does not fit in a model's context window. Splitting it naively loses the fields that only make sense across the whole document, such as totals that reconcile line items from several pages.

   What I did

   A token-budget-aware map-reduce chunker: chunk to a budget, extract per chunk, then reduce the chunk results into one record. A bounded-concurrency batch runner sits over it so a bulk run over a large document set does not exhaust rate limits or memory.

4. 04

   The service accepts webhooks from another team's system. An unauthenticated webhook endpoint is an open door into the extraction pipeline, and auth requirements differed across environments, which is the usual origin of a service that quietly runs open in one of them.

   What I did

   Authentication strategies are selectable at runtime behind a single interface, including HMAC-SHA256 signature verification with constant-time comparison to avoid leaking the signature through response timing. Missing or malformed configuration fails closed instead of falling back to accepting the request, and eight end-to-end tests cover each failure mode rather than only the happy path.

• Integrated into a cross-team Kafka event pipeline against a documented, versioned event contract.
• Webhook authentication layer covered by 8 end-to-end tests spanning every failure mode.

• Python
• LangGraph
• Pydantic
• FastAPI
• Kafka (aiokafka)
• Docker
• Gradio
• HMAC-SHA256

This is commercial work, so there is no public repository or demo to link. I am happy to walk through the architecture, the evaluation harness, or any decision on this page in a call.