Baseten's elephant-1 Beats Reasoning Models by 10x on Classification Tasks
Baseten's Base Labs open-sources a 9B decision model that returns calibrated probabilities instead of text, topping its weight class on the Jev Decision Index.
- Base Labs released elephant-1, a 9B MIT-licensed decision model post-trained from Qwen3.5-9B.
- Returns calibrated probabilities over choice, score, and boolean questions instead of generating text.
- Scores 62.65 on Jev Decision Index 0.4, the best model under 15B, tied #8 of 208.
- Ties d1-ultra on the Artificial Analysis Decision Index replica at 65.2.
- Key tricks: block-bidirectional attention mask, 6.21M curated examples, Brier calibration loss, trained abstention option.
- Free for one week on Baseten Model APIs; weights and technical report open.
Base Labs, Baseten’s research arm, has released elephant-1 weights, a 9-billion-parameter model for workflows whose final output is a label, score, or yes-or-no answer. The model returns probability distributions over predefined answers without generating prose.
The release also includes mammoth-1, a 27-billion-parameter sibling, plus MIT-licensed weights and a technical report covering architecture, data, training, and evaluation. For developers using generative models for routing, classification, guardrails, or automated judging, the practical claim is lower latency with probabilities that can drive confidence thresholds and human escalation.
One state, several decisions
The decision-model category gained a common interface after TypeSafe introduced Jev and the System One API. Base Labs’ decision-model overview describes the same pattern: a request contains a state and one or more questions, and the model assigns normalized probabilities to each question’s allowed answers.
| Format | Purpose | Example |
|---|---|---|
| Choice | Select among named options | Which team should handle this ticket? |
| Score | Rate an input on an ordered scale | Can wait, today, or blocking now |
| Noul | Answer a yes-or-no question | Is the customer requesting a refund? |
The self-hosted runtime calls its yes-or-no format noul. Multiple questions can share one prefix prefill, allowing the runtime to reuse computation over the common state.
from qev.engine import DecisionEngine
from qev.schema import SystemOneRequest
engine = DecisionEngine(
"baselabs/elephant-1",
device="cuda:0",
)
questions = {
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?",
},
"team": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "...",
"technical": "...",
"fraud": "...",
},
},
"urgency": {
"type": "score",
"criteria": ["Can wait", "Today", "Blocking now"],
},
}
request = SystemOneRequest(
state="I was charged twice...",
questions=questions,
)
response, _ = engine.decide(request)
print(response.answers)This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
