Baseten's mammoth-1 Tops 208 Models With 10x Faster Open-Weight Decisions
Base Labs post-trained Qwen3.5 into two open-weight models that output calibrated probabilities over discrete choices instead of generated text.
- Base Labs released mammoth-1 (27B) and elephant-1 (9B), MIT-licensed decision models
- mammoth-1 scores 71.24 on Jev Decision Index 0.4, #1 of 208 models
- elephant-1 ties d1-ultra on Artificial Analysis Decision Index, best under 15B
- Returns calibrated probabilities over typed questions in one shared-prefix forward pass
- Built from Qwen3.5 with block-bidirectional attention and 6.21M curated examples
- Free on Baseten Model APIs for one week with zero data retention
Base Labs, Baseten’s research arm, has released two open-weight models built for discrete decisions: mammoth-1, with 27 billion parameters, and elephant-1, with 9 billion. Each model accepts a shared state, such as text or an image, plus a set of typed questions. It returns a probability distribution over the permitted answers to every question in one forward pass.
The interface follows the decision-model format that TypeSafe introduced with Jev in September 2026. A calibrated probability has operational meaning: predictions assigned 80% confidence should prove correct about 80% of the time across a representative sample. General-purpose chat models can approximate this workflow by generating structured text, but that process adds output tokens, schema validation, retries, and often unreliable confidence scores.
A lead on the public benchmarks
Published results place mammoth-1 first among 208 models on version 0.4 of the Jev Decision Index. Its Full score of 71.24 is 1.82 points above Perplexity Decider v1.2. The benchmark covers knowledge and reasoning, language, tools and automation, retrieval and classification, and arts and human taste; mammoth-1 leads four of those five categories on the public split.
| Model | Parameters | Reported result |
|---|---|---|
| mammoth-1 | 27B | 71.24 and first overall on Jev Decision Index 0.4 |
| Perplexity Decider v1.2 | Not specified | 69.42 on Jev Decision Index 0.4 |
| Cloudflare Clef | Not specified | 55.70 on Jev Decision Index 0.4 |
| elephant-1 | 9B | Highest-scoring 9B model on Jev; tied with d1-ultra on the Artificial Analysis Decision Index |
Elephant-1 scored 62.65 on the Jev Decision Index 0.4, below Jev 1.14 (62.2) and Perplexity Decider v1.2 (69.42). These figures establish performance on the named test suites. Production accuracy and calibration will depend on each application’s data, answer schema, and traffic distribution.
One attention change reshapes inference
Both releases are full fine-tunes of Qwen3.5, using the 27B and 9B variants respectively. Base Labs froze the vision encoder, preserving support for inputs that combine images and text, while training the remaining model weights for the decision interface.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
