Bespoke Labs' Nimble Beats a 27B Model at Classification Using Just 9B
Bespoke Labs open-sources Nimble, a 9B model that makes typed decisions without generating tokens, using contrastive pairs for training.
- Bespoke Labs released Nimble, an open recipe for typed decision models inspired by TypeSafe's Jev.
- Bespoke-Nimble-9B hits 90.12% agreement vs Jev's 93.21% on a 324-example holdout.
- Trained with LoRA on Qwen3.5-9B using only 2,676 curated contrastive pairs, one epoch.
- Scores answer tokens directly from logits, no JSON or chain-of-thought generation needed.
- Runs locally on Apple Silicon via MLX or NVIDIA GPUs via CUDA, no API dependency.
- Contrastive curation flips one focus fact per pair to teach the model what evidence actually matters.
Nimble trains a 9B model for fast, typed classification
Bespoke Labs has released the Nimble repository, an open-weight recipe for typed classification without generated chain-of-thought. The project positions Nimble as an open alternative to TypeSafe’s Jev. Its LoRA adapter, trained on 2,676 examples, reaches 90.12% agreement on a narrow held-out benchmark, compared with 93.21% for Jev.
Typed output from candidate logits
The model weights build on Qwen3.5-9B and target one task: given text and a schema of questions, return typed answers with candidate probabilities. Schemas must be flat, with no nested fields. Each field accepts either a Boolean or an enum containing a fixed set of strings.
Nimble maps every allowed answer to a single vocabulary token and reads the model’s logits for those tokens. Softmax converts the logits into a probability distribution, and Python assembles the typed response. The model generates neither JSON nor explanatory text, which removes parsing failures and autoregressive decoding from the request path. Ordered ratings can also produce an expected value from the candidate distribution.
The repository includes two runtimes. On a Mac, ParallelScorer processes the shared context once and scores the fields in parallel. The CUDA scorer processes each field independently with the complete prompt, so its compute cost grows with the number of fields. Both implementations return the selected answer, raw candidate logits, and normalized probabilities.
Pairs that teach the decision boundary
The released objective uses no teacher probabilities, so Bespoke shapes the model’s decision boundary through a method called contrastive data curation. Each pair contains two nearly identical examples whose labels differ because one relevant fact changes. The question, policy, and unrelated evidence remain fixed, forcing the model to associate the changed fact with the changed decision. The pipeline applies four checks to Choice, Boolean, and Score tasks:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.