Skip to main content
Back

Frankfurt Builds VisualStemma to Trace Manuscripts Without Transcription

A benchmark of 10+ OCR systems on handwritten Old Church Slavonic, plus a novel image-only pipeline that reconstructs manuscript family trees without any transcription.

Pro

  • Benchmark of 10+ OCR systems on handwritten 18th-century Old Church Slavonic manuscripts totaling ~6,000 characters.
  • LightOnOCR-2-1B led at ~6.8% CER; a better prompt cut Gemini 3 Flash from 10.1% to 6.1%.
  • Diacritics remain unsolved, with 70-95% error rates across nearly every model tested.
  • Agentic RAG using retrieved (image, gold, OCR) triples cut CER from 0.258 to 0.151 with no deterioration.
  • New visual stemma method uses ResNet-18 embeddings, k-means, and Hungarian matching to build trees without transcription.
  • Code released at VisualStemma and ocsocr repositories.

Getting a computer to read 18th-century handwritten Church Slavonic is a niche problem, and it turns out to be a strong stress test for the current OCR stack, from classical tools like Tesseract up to frontier vision-language models. A new study from the University of Frankfurt runs that gauntlet, then goes further by introducing a method that reconstructs a manuscript's copy history using only image processing, with no transcription required.

Why handwritten Slavonic breaks OCR

Stemmatology is the discipline of reconstructing the copy history of a text: which manuscript was copied from which, going back centuries. It usually needs high-quality transcriptions of every witness, which is expensive when experts are rare and manuscripts are damaged. The obvious shortcut of running OCR and comparing outputs has a nasty failure mode. OCR errors are systematic, so machine transcripts of different manuscripts end up looking artificially similar to each other, hiding the real relationships.

The author built a small test corpus of roughly 6,000 characters from three 18th-century Valamo monastery manuscripts, then benchmarked more than ten OCR and HTR (handwritten text recognition) systems on it. The lineup covers three families: classical OCR (Tesseract), specialized HTR models (Transkribus, Kraken, TrOCR), and general-purpose vision-language models (GPT-5, Gemini 3 Flash, Qwen3-VL, LightOnOCR).

Old Church Slavonic manuscript page with red and black ink

The scoreboard

Focusing on basic letter accuracy (character error rate after normalizing case, whitespace, and diacritics), the best base systems landed here:

  • LightOnOCR (1B parameter VLM fine-tuned on OCS line images) hit roughly 6.8% CER
  • Gemini 3 Flash reached about 10.1% CER out of the box
  • Transkribus Ukrainian CS models sat around 17%