Sharp-Spark Squeezes a 4B Coding AI Into 6 GB GPUs
A tuned imatrix GGUF repack of Spark-X2.5-4B lands a 4B coder on 6 GB GPUs with usable speed at 131k context.
- Sharp-Spark-X2.5-4B-GGUF is an imatrix-quantized repack of Spark-X2.5-4B targeting 6 GB VRAM GPUs.
- Q6 ties upstream Q8 on MMLU-Pro while using 43% fewer tokens at the median.
- Ties Anthropic Haiku 4.5 on an SWE-bench-Live slice at 4B parameters.
- Hybrid attention: 9 full-attention layers plus 27 windowed, enabling 131k context on small GPUs.
- Patched Sharp-Spark chat template fixes system prompt, tool block, and reasoning-field handling.
- Apache 2.0, runs on recent llama.cpp with spark2_5 support.
A new quantized model called Sharp-Spark repackages Spark-X2.5-4B for local coding workloads. The build combines a custom importance matrix, per-tensor bit allocation, and a patched Jinja chat template. Its repository lists an Apache 2.0 license.
GGUF is the model format commonly used by llama.cpp for local inference. Sharp-Spark’s practical pitch is memory efficiency: its Q6 file occupies 3.61 GB, leaving part of a 6 GB GPU available for context and runtime overhead while retaining scores close to the upstream model’s larger Q8 build.
Small benchmark, close scores
The repository reports results from limited slices of MMLU-Pro and SWE-bench-Live. Spark-Q8 refers to the upstream model’s 8-bit quantized build.
| Test | Reported result | Limit |
|---|---|---|
| MMLU-Pro | Sharp-Spark Q6 scored 69.0%, compared with 70.3% for Spark-Q8. Q6 used 43% fewer tokens at the median and 18% fewer at the mean. | 100 questions |
| SWE-bench-Live | Q6 tied with Claude Haiku 4.5 and Spark-Q8 within the reported confidence intervals. | The summary provides neither the slice size nor exact solve rates. |
The MMLU-Pro difference falls within the reported statistical uncertainty. The SWE-bench-Live comparison carries more uncertainty because the available summary omits its sample size and raw results. These slices support a narrow efficiency claim; broader coding quality still requires full-suite testing on representative repositories.
The token reduction also reflects package behavior beyond quantization. Sharp-Spark injects a terseness instruction by default, which can shorten responses independently of weight precision.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
