webAI's TwIL-LM3 Beats a 120B Model Running on an iPhone
webAI's 3B formal-logic model matches an OpenAI 120B on four of five reasoning lanes and runs on a Raspberry Pi
PRO- webAI released TwIL-LM3, a 3B formal-logic model built on SmolLM3-3B
- Beats gpt-oss-120b on 4 of 5 formal-reasoning lanes at 40x fewer parameters
- Produces 32.9 answers per second, roughly 8x the 120B's throughput, at 482-token generations
- Runs on CPU or 4 GB VRAM via 1.78 GiB Q4_K_M GGUF; targets FOL, entailment, Lean, semantic parsing
- Trained via LoRA SFT, checkpoint fusion, WiSE-FT interpolation, and entropy-weighted GRPO
- Weights available but under non-commercial license, commercial use requires separate agreement
A small model beating a model 40 times its size on any benchmark used to be a headline you'd read once, roll your eyes at, and move on. The math rarely survives the fine print. webAI's new TwIL-LM3 is one of the more interesting cases in recent memory, because the model card itself does the caveating for you, and what remains after the caveats is a genuinely useful result for anyone shipping structured reasoning on device.
webAI released TwiL-LM, a family of small formal-logic models at 1.7 billion and 3 billion parameters that run entirely on consumer hardware. TwiL-LM3, the 3B model, beats gpt-oss-120b on four of five benchmarks in webAI's formal-reasoning suite. The announcement frames this as a bet on specialization over scale, and the numbers back it up in a specific domain.
What it actually does
TwiL-LM translates plain English into formal logic, checks whether conclusions follow from their premises, and works through multi-step deductive reasoning, the backbone of compliance rules, contract conditions, research, and everyday decision-making. This is a contribution to the sub-field known as autoformalization. If you have ever tried to get a general-purpose LLM to reliably output a well-formed first-order-logic translation or a Lean theorem statement, you know why this niche exists.
The model targets five concrete objects rather than prose:
- First-order-logic translation from English
- Entailment labeling (does the conclusion follow?)
- Semantic parsing
- Lean formalization (turning statements into Lean 4 theorem syntax)
- Lean proof critique
The 40x claim, unpacked
The topline number is real but narrow. On webAI's in-domain formal-logic suite, TwIL-LM3 leads gpt-oss-120b on four of five lanes. It also beats every open model up to and including LFM2.5-8B-A1B on all six objective lanes at roughly a third of the parameters. Two larger models still win the overall picture. Qwen3-8B takes the gate 0.5336 to 0.4218, but most of that is loose-match credit; under strict-7 the two sit at 0.2093 and 0.1971. gpt-oss-120b takes the six-lane average 0.5192 to 0.4488.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
