Google's BudouX Fixes Broken Asian Text Wrapping in 15 KB
Google's BudouX brings ML-powered line breaking to CJK languages, now baked into Chrome, Android 14, and the ICU standard library
PRO- BudouX is Google's open-source ML library that predicts natural line break points in Japanese, Chinese, and Thai text.
- It fits in under 15 KB including its ML model, making it viable for client-side use in browsers.
- Uses AdaBoost to classify each character boundary as a break point or not, based on surrounding characters and Unicode blocks.
- Chrome 119+ supports
word-break: auto-phraseCSS property powered by a C++ port of BudouX for Japanese. - BudouX was integrated into ICU (International Components for Unicode), meaning it ships with Android 14 and other ICU-based platforms.
- Available free under Apache 2.0 for Python, JavaScript, and Java; supports custom model training for any language.
BudouX is Google's open-source machine learning library that solves one of the oldest unsolved problems in web typography: where to break lines in Japanese, Chinese, and Thai text. Unlike English, these languages have no spaces between words, so browsers have historically broken lines at arbitrary character boundaries, often mid-word, producing text that looks broken and reads awkwardly. BudouX fixes this by predicting semantically correct break points, and it does so in under 15 KB.
A problem older than the modern web
Traditional text-wrapping algorithms rely on whitespace, which is absent in languages like Japanese, Chinese, and Thai, leading to broken words and poor readability in narrow containers. The original solution, a library called Budou, worked by calling out to third-party word segmenters. The biggest barrier was that it depended on third-party word segmenters, which are large programs infeasible to download for every web page request, and making a request to a cloud-based word segmentation service for every sentence was undesirable considering speed and cost.
BudouX is the successor, designed to eliminate that dependency entirely. It takes only around 15 KB including its machine learning model, making it reasonable to use even on the client-side. That is a remarkable constraint to hit for a model that handles the full complexity of Japanese phrase segmentation.
How it works under the hood
BudouX frames line breaking as a binary classification problem: for every character boundary in a sentence, should a break occur here or not? The library utilizes an AdaBoost-based training approach to build strong classifiers from simple feature stumps for phrase segmentation. AdaBoost (Adaptive Boosting) is an ensemble method that combines many weak classifiers, each looking at a simple feature, into a single strong one. Here, those features are the characters surrounding each potential break point, their Unicode block categories, and combinations of both.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
