ETH Zurich Teaches AI to Read Engineering Drawings With 74.8% Accuracy
A new perspective reframes document recognition as transcription to structured records, enabling the first end-to-end model for engineering drawings.
PRO- Paper reframes document recognition as document-to-record transcription, grouping document types by their intrinsic graph structure.
- Introduces remaining-node prediction, a new inductive bias for set and graph outputs, adapted from a standard transformer backbone.
- First-ever end-to-end model for mechanical engineering drawing transcription, achieving 74.8% exact-match accuracy on simplified L-shapes.
- Sheet music transcription hits 96.6% accuracy, shape drawings from the ABC CAD dataset reach 74.9%.
- Ablations show wrong inductive bias collapses performance to 0%, isolating the source of the gains.
- Published in IJDAR; no code or weights released with the preprint.
Most modern document recognition systems still treat pages as if they were natural photos: run object detection, extract regions of interest, glue the pieces together with hand-written rules. That approach works for text and sheet music, but breaks down on things like engineering drawings, floor plans, and circuit schematics, where the whole point of the document is a web of relationships between symbols. A new paper from Zurich University of Applied Sciences and ETH Zurich argues that this failure is structural, not just a matter of scale, and proposes a fix.
Their reframing is simple. Document recognition should be seen as a transcription task from a document to a record, which implies a natural grouping of documents by the intrinsic structure of their transcription. Under that lens, sheet music and plain text are secretly the same problem (both are sequences), while floor plans and engineering drawings are also the same problem (both are unrestricted graphs). The architecture should reflect that.
Why OCR-style pipelines hit a wall
The authors distinguish between two very different tasks. Photos of the real world contain more information than any model can fully describe, so vision systems settle for partial descriptions like bounding boxes or segmentation masks. Documents are the opposite: humans designed them to encode a specific, finite set of information precisely. Detecting symbols in isolation misses the essential relationships between them, which is exactly what defines meaning in a schematic or a mechanical drawing.
The paper introduces the term
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
