Google DeepMind's SL2T Doubles Sign Language AI Accuracy, Giving Deaf Users a Voice
Google DeepMind's SL2T model brings real-time ASL-to-English dictation to Gboard and Live Transcribe on Pixel 11, trained on 100,000+ hours across 50+ sign languages.
- SL2T launches on Pixel 11: Google DeepMind's sign language-to-text model powers ASL dictation in Gboard and Live Transcribe at no extra cost.
- State-of-the-art benchmark: SL2T scores 70 BLEURT zero-shot on FLEURS-ASL, far above any previously published result.
- 100,000+ hours of training data across 50+ sign languages, with ~25% in ASL; multilingual training outperforms single-language models.
- Privacy-first architecture: MediaPipe Holistic runs on-device to extract pose landmarks; only coordinates (no video) are sent to servers for translation.
- Gloss-free translation: SL2T maps body pose sequences directly to text, bypassing intermediate gloss annotations that limit vocabulary and nuance.
- ASL-only at launch; Google plans to expand to more sign languages and devices, guided by its AI Sign Language Advisory Committee.
For decades, voice-to-text has been a standard feature on every smartphone. Deaf and hard of hearing users have had no equivalent. That changes today: Google DeepMind has announced SL2T (Sign Language to Text), a massively multilingual sign language translation model that powers new sign-to-text dictation features in Gboard and Live Transcribe, launching first on the Pixel 11 family with American Sign Language (ASL) to English support.
What it does and how to use it
Google has embedded the model inside Gboard and Live Transcribe, where it allows Deaf and hard of hearing users to sign to their phone anywhere they would have normally typed. That means signing to search the web, draft messages, ask Gemini a question, or respond in a Live Transcribe conversation, all without touching the keyboard. In practice, this gives users a faster, more natural way to search the web, write messages, and converse with Google's Gemini chatbot.
Sign-to-Text lets you sign into the front camera while the viewer sees the transcription on the outer display -- a particularly useful setup for the Pixel 11 Pro Fold. The feature is available at no additional cost, and Google plans to expand it to more Android devices beyond the Pixel 11 line.
The technical challenge nobody solved until now
Sign language AI has been a notoriously hard problem, and not just for the obvious reasons. AI's ability to process spoken languages has advanced rapidly over recent decades, enabling automatic translation, dictation, and conversational interfaces that feel effortless to hearing users -- yet this technological revolution has not reached the world's more than 200 sign languages and the estimated 70 million Deaf and hard of hearing people who use them.
Two core problems have blocked progress. First, sign languages are not simply encoded versions of spoken languages. Transcribing speech is a matter of performing a sequential mapping from sound to text in the same language, whereas sign languages are independent, natural languages with their own distinct grammars and lexicons -- requiring true machine translation rather than a sequential process of sign-to-word transformations. Second, the model must learn to see and understand physical movement: sign languages convey meaning through simultaneous movements of the hands, arms, torso, head, and face, and accurately tracking these at high frame rates is a difficult and computationally demanding computer vision task.