Anime-Girl 1.5B Gets Imatrix GGUF Builds for CPU Inference

Quantization maintainer mradermacher has published more than 20 imatrix-weighted GGUF versions of Anime-Girl 1.5B. The release lets developers run the conversational model through llama.cpp, Ollama, LM Studio, Jan, and other GGUF-compatible tools without preparing their own quantization pipeline or using a discrete GPU.

The upstream checkpoint uses the Qwen2 architecture and contains roughly 1.5 billion parameters. Its fine-tuning targets anime-style persona chat, placing it among the smaller models suitable for local character dialogue, prototypes, and constrained devices. The GGUF repository ranges from a 437 MB 1-bit build to a 1.27 GB 6-bit version.

How the imatrix preserves useful weights

A plain GGUF conversion can quantize weights using their numeric values alone. An importance matrix adds calibration data gathered by running sample prompts through the model and measuring which weights contribute most strongly to its activations. The quantizer uses that signal when choosing scales and rounding values, reducing error in influential parts of the network.

Calibration can improve low-bit quants, especially the IQ formats, although results depend on how closely the calibration prompts resemble the intended workload. The repository’s quality descriptions come from the publisher rather than an independent benchmark, so developers should compare persona consistency, response quality, latency, and memory use on their own prompts.