Conway Research Squeezes Saluki's 27B Model Into 8 GB and Beats the Original
A 7.89 GB IQ2 quant of Qwen3.8-27B that beats the full 54 GB model on tool-calling benchmarks while fitting on consumer GPUs.
- Conway Research released Underdog Saluki 27B 1.0, a 7.89 GB IQ2 quant of Qwen3.8-27B.
- Scores 88/120 on Berkeley Function Calling tasks versus 84 for the full 54 GB model.
- Parallel tool calls: 42 versus 35 for full size; retains 96% average across 9 benchmarks.
- Runs in stock llama.cpp, Ollama, LM Studio, vLLM; Apache 2.0 licensed and free.
- Weak spots: competition math drops 15-18%, letter-level puzzles, occasional formatting slips.
- Optional 629-928 MB mmproj vision add-on enables image input.
Saluki packs a 27B model into 7.89 GB and keeps tool calling
Conway Research has released Underdog Saluki, a 2-bit GGUF quantization of Qwen3.8-27B. The 7.89 GB model runs in unmodified llama.cpp and, according to Conway’s tests, completes more tool-calling tasks than the 54 GB full-size baseline. Its Hugging Face page had recorded more than 15,000 downloads when this article was prepared.
The release targets local agents on consumer hardware. Aggressive quantization can disrupt the exact token sequences required for function names, JSON arguments, and parallel calls. Saluki uses calibration data selected to preserve that behavior while reducing storage and memory requirements.
Tool calls hold up; math gives ground
Conway evaluated Saluki on Underdog Bench, a fixed 120-task subset of the BFCL v4 leaderboard. The model passed 88 tasks, compared with 84 for the full-size Qwen3.8-27B baseline. Conway also reports that Saluki retains 96% of the parent model’s average performance across nine benchmarks.
| Benchmark | Saluki 2-bit | Full-size model | Difference |
|---|---|---|---|
| Underdog Bench | 88 of 120 | 84 of 120 | +4 tasks |
| Parallel tool calls | 42 | 35 | +7 points |
| IFEval prompt-loose | 93.5 | 91.5 | +2.0 points |
| IFBench prompt-loose | 72.7 | 71.0 | +1.7 points |
| SWE-bench Verified, 50 issues | 30 fixed | 33 fixed | -3 issues |
| MBPP+ | 78.0 | 83.9 | -5.9 points |
| AIME 2025 avg@4 | 79.2 | 96.7 | -17.5 points |
| AIME 2026 avg@4 | 80.0 | 94.6 | -14.6 points |
The release also compares Saluki with the 5.95 GB Bonsai 2, which passes 70 of the 120 tool tasks. Saluki uses about 1.94 GB more storage and passes 18 additional tasks in Conway’s harness.
The per-benchmark results place most of the accuracy loss in competition math and code generation. Instruction following remains close to the baseline, while the reported tool-call scores improve. Public baseline scores and release-run results may use different software or sampling settings, so small differences should be treated cautiously.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
