Peking University Fixes the Hidden Flaw Skewing Diffusion Model Training
New method estimates the unknown floor of diffusion training loss, exposing where models still underfit and improving FID by up to 25%.
Takeaways
- Diffusion loss floor is non-zero and dataset-dependent, making training curves ambiguous without a reference.
- New cDOL estimator computes the optimal loss in linear time with controlled bias and variance.
- Training schedule based on the loss gap improves FID by 2-25% across CIFAR-10, ImageNet-64, and ImageNet-256.
- ImageNet-256 with LightningDiT: baseline 1.42 FID drops to 1.30 with classifier-free guidance.
- Modified power law J(F) - J* = beta * F^alpha fits diffusion scaling far better, correlation 0.82 to 0.99.
- Code and checkpoints promised on acceptance; algorithm is fully specified in the paper's appendix.
Training a diffusion model gives you a loss curve that slowly flattens, but nobody actually knows what number it should flatten toward. Unlike classification, where zero loss is the theoretical floor, diffusion loss bottoms out at some unknown positive value that depends entirely on the dataset. That gap between what a model reports and what an oracle denoiser could achieve has been invisible, and a new paper argues that ignoring it has quietly biased how the field designs training schedules and studies scaling laws.
Researchers from Peking University and Zhongguancun Academy published Diagnosing and Improving Diffusion Models, deriving the optimal loss in closed form and building a scalable estimator that works on datasets like ImageNet. Plugging that estimate back into training design yields FID improvements of 2 to 25 percent across CIFAR-10, ImageNet-64, and ImageNet-256, and makes diffusion scaling curves finally look like clean power laws.
Why the loss floor sits above zero
When you train a diffusion model to predict clean data from noisy data, the target you are actually learning is a conditional expectation, not the original sample. For a given noisy input, many clean sources could plausibly have produced it, so even at optimality, the loss still carries a conditional variance term. That variance depends on the dataset, and it grows with the noise scale. At small noise the floor is near zero; at large noise it approaches the raw data variance.