Kindling AI Shrinks GLM-5.3's 744B Expert Weights to Fit Memory-Constrained Clusters

Kindling AI has published a draft quantization pack for Z.ai’s 744-billion-parameter GLM-5.3 mixture-of-experts model. The Kindling AI pack, named glm-5.3-exlr8-k2-k3-k4, uses EXLR8 v2, a format derived from turboderp’s EXL3. It targets clusters of NVIDIA GB10-class devices and reduces the expert-weight data each node must stream while generating tokens.

GB10-class nodes combine 128 GB of unified LPDDR5X memory with roughly 220 GB/s of bandwidth. GLM-5.3 routes each token through a subset of its experts, so repeatedly loading the selected weights can dominate decode time, especially above batch size one. Even the smallest EXLR8 manifest contains more routed-weight data than one 128 GB node can hold, making multi-node tensor parallelism central to the design.

The draft still has open gates

The repository identifies the release as a preview with several acceptance steps unfinished:

• Complete: The distributed quantization encode.
• In progress: Assembly of the final file set.
• Pending: Agreement across tensor-parallel configurations, held-out quality checks against BF16, and decode-speed measurements.

The current repository exposes encoded artifacts, though its final quality and throughput claims remain provisional. Tensor-parallel agreement checks whether different device splits produce consistent results. The held-out test measures divergence from the original BF16 model, and the decode benchmark measures token-generation speed after prompt processing.

Four changes target bandwidth

EXLR8 v2 retains EXL3’s trellis coding, which searches a constrained sequence of quantized values to minimize error. EXL3 itself adapts ideas from Cornell RelaxML’s QTIP work. Kindling modifies the codec and storage layout for large MoE models: