University of Zurich's RUMLEM Beats ML Classifiers on a Nearly Invisible Language
A dictionary-driven lemmatizer covers all six Romansh varieties, hitting 80% word coverage and 95% variety identification accuracy on 30,000 texts.
ProTakeaways
- RUMLEM is a dictionary-based lemmatizer covering all six Romansh varieties, released open source on GitHub.
- Built from Pledari Grond, yielding 725,005 word forms mapped to 178,467 lemmas across varieties.
- Lemmatizes 77 to 84% of tokens in typical Romansh text and returns full morphological features plus German glosses.
- Identifies the correct Romansh variety with 95% accuracy on 30,000 texts, hitting 100% on texts over 300 tokens.
- Cleanly separates Romansh from French, Italian, Catalan, and Romanian using a simple word-recognition threshold near 0.6.
- Try it via
pip install rumlemor the Hugging Face demo.
Romansh is one of Switzerland's four national languages, but it barely exists in modern NLP toolchains. A team at the University of Zurich just released RUMLEM, a dictionary-based lemmatizer that finally gives the language proper morphological tooling, and as a bonus, doubles as a variety and language identifier that beats machine-learning baselines on short texts.
A language with five siblings
Romansh is a minority Romance language spoken by roughly 40,000 to 60,000 people across several Alpine valleys of Switzerland. It comprises five regional varieties, or idioms (Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader), plus Rumantsch Grischun, the supra-regional standard. The varieties diverge enough that mutual intelligibility is often limited, which is why variety-specific tooling matters.
Building one Romansh model is really building six, and until now the language has been mostly invisible to mainstream infrastructure. Romansh and its varieties are absent from popular NLP resources such as SpaCy, Universal Dependencies, and UniMorph. Prior work has tackled contextual embeddings, named entity recognition, and word alignment, but lemmatization, the task of mapping inflected word forms back to their dictionary entry, was still missing.
Why dictionaries instead of a neural model
Neural lemmatizers usually need annotated treebanks, which do not exist for Romansh. So the authors went the other direction. Dictionary-based lemmatization is a viable alternative when a comprehensive dictionary is available, and has been used for German (via Wiktionary), Middle English, Latin, and Somali.