Scantist Shrinks GLM-5.3 by 25% to Fit on a Single NVIDIA B200
A quantized GLM-5.3 variant trims 64 experts via neural architecture search, shrinking the model 24.6% so it fits on a single B200 GPU.
- New GLM-5.3-SLIM-GGUF fits a GLM-5.3-class model on one NVIDIA B200 at 179.9 GB.
- Neural architecture search removed 64 of 256 routed experts per MoE layer, no retraining required.
- Scores 92.7% HumanEval pass@1 and 84.8% GPQA-Diamond with reasoning on.
- Parameter count drops from 753B to 563B, roughly 2.56 bits per weight.
- Reaches 203 tok/s total generation at 64 parallel sequences on one B200.
- Built on Unsloth's UD-IQ2_M quantization of zai-org/GLM-5.3; runs via llama.cpp, Ollama, LM Studio.
GLM-5.3-SLIM-GGUF reduces a 753-billion-parameter mixture-of-experts model to 563 billion parameters, allowing the quantized weights to run on a single NVIDIA B200. A neural architecture search selected 192 of the 256 routed experts in each MoE layer, and the conversion removed the remaining 64 without retraining or recalibration.
The release combines Unsloth’s UD-IQ2_M quantization of GLM-5.3 with the expert selection from GLM-5.3-SLIM-E192. GGUF is a model format designed for runtimes such as llama.cpp; here, its tensor layout makes it possible to delete complete experts while preserving the selected quantized weights byte for byte.
A 58.7 GB cut
| Model | Experts per MoE layer | Parameters | GGUF size |
|---|---|---|---|
| GLM-5.3 UD-IQ2_M | 256 | 753B | 238.6 GB |
| GLM-5.3-SLIM-GGUF | 192 | 563B | 179.9 GB |
| Reduction | 25% | 25.2% | 24.6% |
The slim release averages about 2.56 bits per weight. Its 238.6 GB parent exceeds the memory available on one B200 and a 192 GB Mac, forcing multi-device inference. On the tested B200, the authors report that the pruned model remains entirely on the GPU while leaving enough memory for roughly 100,000 tokens of shared key-value cache across as many as 10 concurrent requests.
Scores without a matched baseline
| Benchmark | Result | Evaluation setup |
|---|---|---|
| HumanEval pass@1 | 92.7% (152/164) | Greedy decoding; thinking disabled |
| GPQA-Diamond | 84.8% (168/198) | reasoning_effort=high; 65,536-token generation budget |
| MMLU subset | 71.4% | Zero-shot; 570 questions |
A matched run of the 256-expert parent is required to quantify the quality lost through pruning and quantization. The harness also differs from the official GLM-5.3 evaluation, so these figures should be interpreted within the published setup rather than compared directly with the model card.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
