Thinking Machines' Inkling 2 Mini Loops Layers to Beat Larger Open Models
Thinking Machines releases a 26B looped-transformer MoE that runs on a laptop, posts 20.1 on AA Index, and ships with open weights for monitorability research.
Takeaways
- Thinking Machines released Inkling 2 Mini, a 26.4B MoE looped transformer with 3.4B active parameters.
- Middle layers run three times per token, giving 48 layer visits, matching Inkling 2 Small's depth.
- Scores 20.1 on AA Index, 48.2% on DeepSWE v1.1, 77.9% on SWE-bench Pro v2 Hard.
- ~20GB footprint in native NVFP4/MXFP8 runs offline on a MacBook via Hugging Face and Ollama.
- Variable-loop research variant TML-d24-varloop-research released for monitorability and interpretability study.
- Trained with NVFP4 end-to-end, expert pruning plus 1T-token healing, and 2.2T-token RL post-training.
Thinking Machines has released Inkling 2 Mini, a 26.4-billion-parameter mixture-of-experts model designed for agentic coding and tool use. The open-weight release includes model weights, inference and fine-tuning through Tinker, and support for SGLang, vLLM, Ollama, Unsloth, and llama.cpp.
Inkling 2 Mini runs its middle transformer layers three times for each generated token. Those repeated passes add sequential computation without adding stored parameters, allowing the model to refine its internal representation before producing the next token.
Forty-eight layer visits from 24 layers
Standard transformers pass each token through a fixed stack of layers once. Inkling 2 Mini divides 24 physical layers into four blocks and repeats the two middle blocks three times. Each token therefore makes 48 layer visits, matching Inkling 2 Small while using about one twenty-second of its total parameters.
Thinking Machines reports that repeating the middle 12 layers outperformed repeating the final 12 layers on 27 of 28 evaluations. The company attributes the result to a division of labor in which middle layers handle more abstract processing while the outer layers translate between embeddings and internal representations.
Attention residuals carry information between passes. A learned gate initially suppresses the previous pass’s contribution, then opens gradually during training. This schedule lets the network learn recurrent computation without destabilizing the loss curve.
The sparse mixture-of-experts design stores 26.4 billion parameters while activating 3.4 billion for each token. Thinking Machines counts 5.4 billion effective weight activations after including the repeated reads created by looping.
Coding leads the scorecard
Thinking Machines says Inkling 2 Mini leads several agentic coding evaluations within its comparison set of open-weight models below 100 billion total parameters.
| Benchmark | Reported score |
|---|---|
| DeepSWE v1.1 | 48.2% resolved |
| SWE-bench Pro v2, Hard | 77.9% |
| SWE-bench Verified | 79.2% |
| Vals Index v2.2 | 46.5% |
| AA Index v5.0 | 20.1, tied with Qwen3.8-27B |
| BrowseComp | 79.0% |
| DeepSearchQA | 85.3% |
| MCP Atlas | 82.7% |
The model also approaches Inkling 2 Small on PMPP-Hard, which tests kernel optimization. Its weaker areas include multi-turn chat, with 50.0% on MultiChallenge, and strict instruction following, with 33.4% on ComplexConstraints. Larger models retain an advantage on intensive reasoning evaluations such as HLE-Diamond and CritPt v2.
Factual recall remains comparable to other models in the 26-billion-parameter class because additional passes increase computation while leaving stored knowledge unchanged. Tool access can supply external information, which helps explain the stronger results on browsing and MCP-based tasks.
Agentic benchmark scores depend heavily on the surrounding scaffold, tool budget, prompts, and evaluation harness. Developers comparing models should reproduce the reported setup rather than treating scores from different harnesses as directly interchangeable.
A 20GB model, plus runtime headroom
Most weights were trained and deployed in NVFP4, while the final four layers use MXFP8. The mixed-precision checkpoint occupies about 20GB, and Thinking Machines recommends at least 32GB of unified memory for Ollama on a Mac to accommodate runtime overhead and context state.
Thinking Machines demonstrated the model generating a playable flight simulator within a few minutes on an M5 Pro MacBook operating offline. A second demonstration used one local model instance to control 20 villagers in a simulated pixel town.
Researchers from Stanford’s Hazy Research and Scaling Intelligence labs measured energy efficiency on an NVIDIA DGX Spark using the methodology from the efficiency paper. Their reported intelligence-per-joule scores were 0.703 for Inkling 2 Mini, 0.430 for Qwen3.6-35B-A3B, and 0.159 for Gemma-4-26B-A4B.
| Runtime | Launch support |
|---|---|
| Tinker | Hosted inference and fine-tuning |
| SGLang and vLLM | Server inference |
| Ollama | Local Mac deployment with 32GB or more memory |
| Unsloth and llama.cpp | Quantized GGUF deployment |
| DGX Spark | Native NVFP4 inference |
Prune, heal, then align
Training began on Thinking Machines’ main scaling pipeline with 384 experts per layer and eight selected for each token, a 48-to-1 sparsity ratio. The company trained the model end to end in NVFP4 using a dedicated kernel on NVIDIA GB300 hardware.
- Expert pruning: Two-thirds of the experts were removed during mid-training, leaving the 128 most frequently used experts in each layer.
- Healing: The pruned model processed one trillion tokens distilled from a larger internal Inkling model, allowing the surviving experts to absorb work previously handled by deleted experts.
- Supervised fine-tuning: The model trained on a data mixture similar to the one used for Inkling 2 Small.
- Reinforcement learning: Training covered 2.2 trillion tokens across 2,000 steps and used on-policy distillation, which applies teacher guidance to trajectories generated by the current model.
The released checkpoint retains the same NVFP4 and MXFP8 formats used during training, avoiding accuracy loss from post-training conversion for the native deployment. Quantized GGUF variants introduce their own precision trade-offs.
More loops, shorter traces
Additional recurrent computation could affect chain-of-thought monitoring because generated reasoning traces expose only written tokens, while intermediate loop states remain internal. Thinking Machines released a research checkpoint trained with one to five passes so researchers can compare reasoning depth under controlled conditions.
- Longer internal task horizons: On Think Fast, the no-chain-of-thought horizon rose from 16 seconds with one pass to 37 seconds with three passes, then plateaued.
- Generalization beyond training depth: N-Hop Lookup performance continued improving through 10 passes, twice the maximum used during training. The benchmark tests serial retrieval across linked facts.
- Shorter reasoning traces: For GPQA Diamond questions answered correctly at every tested depth, traces averaged 3,703 tokens with one pass and 1,661 with five. Sentence length and dictionary-word frequency stayed stable, while backtracking declined.
- No measured rise in stealth: LinuxArena sabotage tests showed no consistent increase in stealth rate as loop count grew. The company describes the finding as preliminary and expects larger models to require further study.
Thinking Machines is also working with the National Deep Inference Fabric and provided Goodfire with early access for mechanistic interpretability research into the recurrent states.
Where the release fits
Inkling 2 Mini gives developers a compact option for coding agents, browsing systems, MCP tools, and offline workflows where local execution, data control, or predictable inference costs matter. Its weaker chat, constraint-following, and factual-recall results make workload-specific evaluation essential before deployment.
Open weights provide access to the checkpoint, while usage and redistribution rights depend on the license published with the model. Teams should review those terms, confirm runtime support for the looped architecture, and test the chosen precision format against their latency and accuracy requirements.
The variable-loop checkpoint also gives researchers a controlled way to study how recurrent depth changes capability and monitorability. The quantization-aware training pipeline and the prune-then-heal recipe remain applicable to other model architectures even when repeated transformer blocks are absent.