Frankfurt Builds VisualStemma to Trace Manuscripts Without Transcription
A benchmark of 10+ OCR systems on handwritten Old Church Slavonic, plus a novel image-only pipeline that reconstructs manuscript family trees without any transcription.
ProTakeaways
- Benchmark of 10+ OCR systems on handwritten 18th-century Old Church Slavonic manuscripts totaling ~6,000 characters.
- LightOnOCR-2-1B led at ~6.8% CER; a better prompt cut Gemini 3 Flash from 10.1% to 6.1%.
- Diacritics remain unsolved, with 70-95% error rates across nearly every model tested.
- Agentic RAG using retrieved (image, gold, OCR) triples cut CER from 0.258 to 0.151 with no deterioration.
- New visual stemma method uses ResNet-18 embeddings, k-means, and Hungarian matching to build trees without transcription.
- Code released at VisualStemma and ocsocr repositories.
Getting a computer to read 18th-century handwritten Church Slavonic is a niche problem, and it turns out to be a strong stress test for the current OCR stack, from classical tools like Tesseract up to frontier vision-language models. A new study from the University of Frankfurt runs that gauntlet, then goes further by introducing a method that reconstructs a manuscript's copy history using only image processing, with no transcription required.
Why handwritten Slavonic breaks OCR
Stemmatology is the discipline of reconstructing the copy history of a text: which manuscript was copied from which, going back centuries. It usually needs high-quality transcriptions of every witness, which is expensive when experts are rare and manuscripts are damaged. The obvious shortcut of running OCR and comparing outputs has a nasty failure mode. OCR errors are systematic, so machine transcripts of different manuscripts end up looking artificially similar to each other, hiding the real relationships.
The author built a small test corpus of roughly 6,000 characters from three 18th-century Valamo monastery manuscripts, then benchmarked more than ten OCR and HTR (handwritten text recognition) systems on it. The lineup covers three families: classical OCR (Tesseract), specialized HTR models (Transkribus, Kraken, TrOCR), and general-purpose vision-language models (GPT-5, Gemini 3 Flash, Qwen3-VL, LightOnOCR).

The scoreboard
Focusing on basic letter accuracy (character error rate after normalizing case, whitespace, and diacritics), the best base systems landed here:
- LightOnOCR (1B parameter VLM fine-tuned on OCS line images) hit roughly 6.8% CER
- Gemini 3 Flash reached about 10.1% CER out of the box
- Transkribus Ukrainian CS models sat around 17%