A document-intelligence demo works on clean, well-formatted PDFs chosen because they work. A document-intelligence system has to work on the scanned, rotated, multi-column, hand-annotated and occasionally corrupted files that a real organisation actually produces. The gap between those two is not a model problem — it is a pipeline problem.
Extraction is the easy 80%
Getting structured fields out of a well-formed document is now a solved problem for most common document types. What determines whether a document pipeline is usable in production is what happens to the other cases: the field that was not present, the value that does not match the expected format, the document that is a photograph of a document.
In practice. Build the exception path before the happy path is polished. A pipeline that confidently extracts wrong values from bad input is worse than one that flags them for review.
Design for three outcomes, not two
Most document pipelines are built as if there are two outcomes: extracted correctly, or failed. Production pipelines need a third — extracted with low confidence, routed to a human — and, critically, the human's correction has to feed back into understanding where the pipeline systematically struggles.
- Confidence scoring per field, not just per document — a document can be 90% right and wrong in exactly the field that matters.
- A review queue that shows the source document alongside the extracted value, not the value alone.
- Aggregated review data treated as a diagnostic signal, not just a correction log.
Validation belongs before the data reaches anything downstream
An extracted value that looks plausible but is wrong is more dangerous than one that visibly fails, because it propagates. Business-rule validation — does this total match these line items, is this date in a sane range, does this identifier match a known format — catches a category of error that model confidence scores do not, and it belongs in the pipeline, not in whatever system consumes the output afterward.
The reliable version is the boring version
A document pipeline that reliably handles messy real-world input rarely looks impressive in a demo — it looks like a well-instrumented state machine with clear exception handling. That is the correct trade: the interesting engineering work is deciding what happens when extraction is uncertain, not extracting the fields that were never going to be difficult.
References
Exception-handling patterns from AFC document-pipeline engagements are available on request.