Ostris Shrinks Ideogram 4's 9B Memory Blocker Into a 109MB LoRA

Ideogram 4 has a quirk that makes it expensive to run locally: it actually uses two completely separate 9.3B parameter transformer models every time it generates an image. One follows your text prompt (the conditional branch), and one runs without any text at all (the unconditional branch) to anchor image quality through classifier-free guidance (CFG) -- a technique where the model steers generation by comparing a text-conditioned output against an unconditioned baseline. Classifier-free guidance in Ideogram 4 is realized by shipping two separate weight copies of the backbone -- a conditional and an unconditional transformer. That means loading the full pipeline means holding two copies of a massive model in memory simultaneously.

Ostris, the developer behind the popular ai-toolkit training framework, just released a 109MB LoRA that replaces the full unconditional model during training. The file is called ideogram_4_unconditional_lora and it ships as a single .safetensors file. It is free, available now on Hugging Face, and already baked into ai-toolkit as the default.

Why the unconditional branch is such a headache

Classifier-free guidance combines a conditional pass (text + image latents) with an unconditional pass. Ideogram 4 makes the unconditional pass asymmetric: it drops the text tokens entirely instead of replacing them with padding, so the unconditional pass runs only over image tokens. This design is clever for inference quality, but it means the unconditional model is a distinct, fully-sized transformer you have to load separately. The conditional and unconditional branches can be independently refined, enabling separate control over prompt adherence and image quality.