AI · Operations
Automation that survives contact with real data
Demos use clean PDFs. Operations uses forwards, scans, and the PDF that is actually a photograph of a whiteboard.
2025-09-18 · 8 min
The accuracy number from a vendor deck was measured on a dataset that looks nothing like your inbox. Production data is hostile: missing pages, mixed languages, someone CC'd the whole company, the CRM field is a paragraph.
Schema first, model second
Decide the record you are willing to store. Validate. If the model cannot fill a field, leave it empty and escalate. Invented values in a system of record are worse than no automation.
Build the review UI
Operators need source spans, diffs, and keyboard speed. If review is a spreadsheet export, you have built a new job, not a system. The queue is the product.
Evaluate on your residue
Keep the ugly cases. Run them when prompts change. A living eval set is less glamorous than a new model name and infinitely more valuable.