Comfy-Org Makes NVIDIA's PixelDiT Run Locally Without a VAE
Comfy-Org repackages NVIDIA's VAE-free PixelDiT model for ComfyUI, making the CVPR 2026 pixel-space diffusion transformer plug-and-play for local image generation.
Takeaways
- Comfy-Org repackaged NVIDIA's PixelDiT weights as
.safetensorsfiles for native ComfyUI use, no custom code needed. - PixelDiT is VAE-free — it generates images directly in pixel space, avoiding lossy compression artifacts that plague latent diffusion models.
- Dual-level architecture: a patch-level DiT handles global semantics; a pixel-level DiT refines per-pixel texture detail.
- 1.3B parameters, 1024px output, consumer GPU-compatible; achieves 1.61 FID on ImageNet 256×256, approaching top latent models.
- Two-file setup: drop
pixeldit_1300m_1024px_bf16.safetensorsandgemma_2_2b_it_elm_bf16.safetensorsinto ComfyUI model folders, load the official workflow template. - Non-commercial license only (NSCLv1); best for research, text-in-image tasks, and fine-detail generation where VAE artifacts are a problem.
Comfy-Org has published repackaged weights for NVIDIA's PixelDiT on Hugging Face, reformatting the model files so they drop straight into a ComfyUI install with no custom code required. The underlying model is a research release from NVIDIA that earned an Oral presentation at CVPR 2026 , and it works in a fundamentally different way from every latent diffusion model you've used before.
No VAE. Seriously.
Almost every popular image generation model today , Stable Diffusion, FLUX, you name it , compresses images into a compact "latent" space using a Variational Autoencoder (VAE), runs the diffusion process there, then decodes back to pixels. That compression step is lossy: fine details like text, sharp textures, and high-frequency patterns get blurred or distorted before the model even starts generating.
PixelDiT is a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in pixel space. The practical upshot: PixelDiT keeps fine lettering intact because it denoises directly in pixel space , no lossy VAE, no baked-in artifacts , while latent models like FLUX smear characters after VAE reconstruction.
The architecture doing the heavy lifting
Working directly in pixel space sounds simple, but it's computationally brutal. Every pixel is a token, and attention over millions of tokens is quadratically expensive. NVIDIA's solution is a two-level design that splits the work intelligently.
- Patch-level DiT: