Skip to main content
Back

Comfy-Org Makes NVIDIA's PixelDiT Run Locally Without a VAE

Comfy-Org repackages NVIDIA's VAE-free PixelDiT model for ComfyUI, making the CVPR 2026 pixel-space diffusion transformer plug-and-play for local image generation.

  • Comfy-Org repackaged NVIDIA's PixelDiT weights as .safetensors files for native ComfyUI use, no custom code needed.
  • PixelDiT is VAE-free — it generates images directly in pixel space, avoiding lossy compression artifacts that plague latent diffusion models.
  • Dual-level architecture: a patch-level DiT handles global semantics; a pixel-level DiT refines per-pixel texture detail.
  • 1.3B parameters, 1024px output, consumer GPU-compatible; achieves 1.61 FID on ImageNet 256×256, approaching top latent models.
  • Two-file setup: drop pixeldit_1300m_1024px_bf16.safetensors and gemma_2_2b_it_elm_bf16.safetensors into ComfyUI model folders, load the official workflow template.
  • Non-commercial license only (NSCLv1); best for research, text-in-image tasks, and fine-detail generation where VAE artifacts are a problem.

Comfy-Org has published repackaged weights for NVIDIA's PixelDiT on Hugging Face, reformatting the model files so they drop straight into a ComfyUI install with no custom code required. The underlying model is a research release from NVIDIA that earned an Oral presentation at CVPR 2026 , and it works in a fundamentally different way from every latent diffusion model you've used before.

No VAE. Seriously.

Almost every popular image generation model today , Stable Diffusion, FLUX, you name it , compresses images into a compact "latent" space using a Variational Autoencoder (VAE), runs the diffusion process there, then decodes back to pixels. That compression step is lossy: fine details like text, sharp textures, and high-frequency patterns get blurred or distorted before the model even starts generating.

PixelDiT is a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in pixel space. The practical upshot: PixelDiT keeps fine lettering intact because it denoises directly in pixel space , no lossy VAE, no baked-in artifacts , while latent models like FLUX smear characters after VAE reconstruction.

The architecture doing the heavy lifting

Working directly in pixel space sounds simple, but it's computationally brutal. Every pixel is a token, and attention over millions of tokens is quadratically expensive. NVIDIA's solution is a two-level design that splits the work intelligently.

  • Patch-level DiT: