Local Grammar Rules Alone Can Trigger a Sharp Order Transition
Physicists built a context-sensitive grammar where only nearby symbols interact, and still found a phase transition, hinting that language itself drives the order.
- New paper builds a context-sensitive random language model with strictly short-range interactions between symbols.
- Numerical Monte Carlo on sentences up to N=4096 shows a clean finite-temperature phase transition.
- Finite-size scaling gives critical temperature k_B T_c = 0.24, with exponents nu = 2.50 and gamma = 2.00.
- Result rules out long-range interactions as the cause of the babbling-to-grammar transition in earlier RLMs.
- Supports the view that context-sensitive grammars are needed for genuine phase transitions in language models.
- Published in Phys. Rev. Research 8, 043073 (2026); preprint on arXiv, no code released.
A study in Physical Review Research addresses a disputed result at the intersection of statistical physics and linguistics. Earlier toy language models appeared to switch abruptly from random output to structured sequences, but their long-range interactions could have caused the effect. The new model restricts each symbol to a fixed local context and still produces a continuous phase transition, showing that long-range coupling is unnecessary.
A disputed jump from babble to order
Eric DeGiuli introduced the 2019 RLM paper, which defined a Random Language Model as an ensemble of weighted context-free grammars. Using methods from spin-glass physics and rule weights drawn from a lognormal distribution, the analysis found a high-entropy “babbling” phase and a lower-entropy phase with structured output. The model offered a stylized account of how syntax might emerge.
A 2022 analysis by Nakaishi and Hukushima found no thermodynamic phase transition in the basic RLM and proposed studying more expressive, context-sensitive grammars. Later numerical studies reported Berezinskii-Kosterlitz-Thouless-like transitions in models where distant symbols interacted. Such interactions can generate collective ordering by themselves, leaving the contribution of grammatical constraints unresolved.
Local context is enough
Toji, Takahashi, Roychowdhury, and Miyahara’s new construction moves from context-free to context-sensitive grammar. Its production rules may depend on neighboring symbols, but the context width remains fixed as sentence length grows. In statistical-mechanics terms, every interaction is short-range.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
