LG’s 4-bit EXAONE 3.5 approaches one million monthly Hugging Face downloads

At publication time, LG AI Research’s AWQ-quantized EXAONE 3.5 7.8B Instruct checkpoint was approaching one million monthly downloads on its AWQ model page. Hugging Face counts qualifying requests, so the figure does not represent one million unique users or deployments. It does indicate sustained interest in a compact Korean-English model designed for local inference.

The wider EXAONE 3.5 family includes 2.4B, 7.8B, and 32B instruction-tuned models. This 7.8B release uses activation-aware weight quantization, or AWQ, to reduce GPU memory requirements while retaining the model’s configured 32,768-token context window.

A compact checkpoint with a large window

The checkpoint contains a decoder-only Transformer with the following specifications:

• 6.98 billion non-embedding parameters across 32 layers
• Grouped-query attention with 32 query heads and 8 key/value heads
• A 102,400-token byte-level BPE vocabulary designed around Korean and English
• A maximum sequence length of 32,768 tokens, including input and generated tokens
• W4A16 group-wise quantization with groups of 128 weights

W4A16 stores quantized weights at 4-bit precision while keeping activations at 16-bit precision. Groups of 128 weights share quantization metadata. The resulting files occupy roughly 5 GB, although actual GPU memory use also includes the key-value cache, activations, temporary buffers, and backend overhead.

Quantization leaves the architecture and configured context length unchanged. A full 32K sequence at batch size one can add roughly 4 GiB for a 16-bit key-value cache before other allocations, so loading the weights in about 5 GB does not mean 32K inference will run within that same memory budget.

EXAONE 3.5 retains the broad architecture of EXAONE 3.0 7.8B. According to LG’s technical report, long-context fine-tuning extended the maximum sequence length from 4,096 to 32,768 tokens. The model also uses a rotary-position-embedding theta of 1,000,000 to support the larger window.