Surya Institute's 86M Model Beats Giants 100x Its Size at Arithmetic
An 86M-parameter model trained on an Indonesian teaching method beats models 100x its size at basic arithmetic, revealing how structured pedagogy shapes neural computation
Takeaways
- An 86M-parameter GPT-2 model trained on Indonesian GASING arithmetic pedagogy beats models 100x its size on basic four-operation arithmetic.
- GASING's left-to-right procedure naturally aligns with autoregressive token generation, making it an ideal training curriculum for language models.
- Training used only next-token prediction -- no reinforcement learning, no reward model, no verifier.
- Mechanistic analysis (attention masking, residual probing, logit lens) reveals three learning phases: procedural internalization, then mental-arithmetic shortcutting.
- The TOBA syllabic tokenizer keeps digit sequences intact, avoiding the fragmentation that makes arithmetic hard for standard tokenizers.
- No model weights or code are publicly released yet; the paper is on arXiv with full methodology details.
What if the secret to teaching a language model arithmetic was to borrow from a classroom in Indonesia? That is the central bet of a new paper from researchers at the Surya Institute, who trained a tiny 86-million-parameter GPT-2-style model to do arithmetic using a human teaching method called GASING -- and watched it outperform models more than 100 times its size.
The result is not just a performance number. The paper offers one of the clearest windows yet into how a language model actually learns to compute, tracing the emergence of arithmetic skill through three distinct phases and linking each phase to measurable changes inside the network.
Why arithmetic is still a hard problem
Language models have a well-known arithmetic problem. LLMs can be surprisingly bad at arithmetic, often due to the arbitrary way digits are tokenized. Standard tokenizers, built to compress English text efficiently, treat numbers inconsistently: the string "128" might become one token, two tokens, or three, depending on context. That inconsistency makes it hard for a model to learn positional relationships between digits, which are essential for carrying, borrowing, and place-value reasoning.
The dominant fix has been scale. Chain-of-thought prompting is an emergent ability of scale -- it requires a sufficiently large language model to improve performance, and actually hurts performance for small language models. That creates a frustrating situation: the models that most need structured reasoning help are the ones that benefit least from it.
The GASING method: a classroom technique repurposed for tokens
GASING stands for Gampang, Asyik, Menyenangkan -- Easy, Fun, Enjoyable -- and covers all four basic arithmetic operations: addition, multiplication, subtraction, and division. It was discovered by Professor Yohanes Surya and has been widely used across Indonesia to address numeracy gaps in under-resourced communities.