Unsloth's Llama 3.2 Mirror Hits 800K Downloads by Removing Meta's Login Wall
Unsloth's reupload of Meta's 1B instruction-tuned Llama ships GGUF and 4-bit quantizations tuned for 2x faster finetuning with 70% less memory.
- Unsloth's Llama-3.2-1B-Instruct mirror hits 802k downloads and 3.6k likes on Hugging Face.
- Repo is ungated, unlike Meta's original, so CI pipelines and Colab notebooks just work.
- Companion GGUF repo ships quantizations from 422 MB (UD-IQ1_S) up to BF16.
- The 4-bit bnb variant enables QLoRA finetuning 2x faster with 70% less memory.
- Base model from Meta uses pruning plus distillation from larger Llama teachers.
- Hosted inference starts at $0.027 per 1M input tokens across six cloud providers.
Why Unsloth’s 1B Llama mirror keeps spreading
Unsloth’s Llama repository has climbed Hugging Face’s trending charts. At publication, the page showed more than 800,000 downloads in its displayed monthly count and 3,600 likes. The weights come from Meta, but Unsloth packages them for easier downloading, quantized inference, and single-GPU adapter training. That distribution work explains the repository’s reach.
Small weights, broad reach
Meta released Llama 3.2 in September 2024 with 11B and 90B vision models and compact 1B and 3B text models. According to the Meta release, the smaller models target mobile devices, edge hardware, retrieval, summarization, and multilingual assistants.
The 1B Instruct model contains about 1.23 billion parameters and was produced through pruning and knowledge distillation from larger Llama models, followed by instruction tuning. Meta reports pretraining on as many as 9 trillion tokens, a December 2023 knowledge cutoff, and official support for English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
Grouped-query attention reduces the memory and computation required during generation by letting multiple query heads share key and value heads. The architecture supports a 128K-token context window, although usable length depends on available memory, inference software, and workload.
One model, several targets
Unsloth preserves the full BF16 safetensors weights, which occupy about 2.49 GB, and publishes sibling repositories for other runtimes. Its GGUF builds support llama.cpp, Ollama, LM Studio, CPU inference, and Apple Silicon. The separate 4-bit build targets CUDA systems and QLoRA training.
Quantization stores weights with fewer bits. It reduces storage and memory use, with some loss of numerical precision and potentially model quality. The available packages cover several deployment needs:
| Package | Typical use | Main trade-off |
|---|---|---|
| BF16 safetensors | Transformers inference and high-fidelity fine-tuning |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
