OrcaRouter Squeezes a 763B DeepSeek-V4.1 Model Into 256 GB Workstations

OrcaRouter has published two GGUF builds of an abliterated DeepSeek-V4.1-Flash-1017. GGUF is the model format used by llama.cpp. The larger build reports 235.8 GB of resident model data and stays close to the unquantized reference in short-context tests, putting a 763-billion-parameter Mixture-of-Experts model within reach of a carefully configured workstation or server. The release also removes much of the model’s refusal behavior, creating deployment risks alongside its value for local inference and security research.

How 763B parameters fit

DeepSeek-V4.1-Flash-1017 uses 384 routed experts and activates six for each token, so inference touches only part of the expert network at a time. Its architecture also includes a 197B-parameter Engram n-gram memory, compressed sparse attention, and a nominal 1-million-token context window.

The Engram component reads 48 lookup rows per token. llama.cpp memory-maps its roughly 110 GB table and loads required pages from storage, allowing the table to remain on an NVMe SSD instead of occupying RAM permanently. That storage plan accounts for much of the gap between the model’s total parameter count and the 235.8 GB allocation reported for the larger build.

OrcaRouter assigns precision separately across layer and tensor groups. Each group receives a quantization level based on measured divergence from the unquantized model, concentrating higher precision where compression changes the output distribution most.

The 128 GB floor costs accuracy

The repository provides two mixed-precision allocations with substantially different fidelity:

Kullback-Leibler divergence, or KLD, measures how far the quantized model’s token probabilities move from the reference distribution; lower values indicate closer agreement. Top-1 agreement records how often both versions select the same next token. Bits per weight, or bpw, applies here to expert weights rather than every component in the checkpoint.

The reported memory figures cover the model allocation under OrcaRouter’s mapping plan. Serving also requires space for the KV cache, compute buffers, the operating system, and runtime overhead. A 235.8 GB allocation therefore leaves limited headroom on a 256 GB host, especially as context length and concurrency increase.