Skip to main content
Back

IAFS Fixes AI Image Upscaling's Biggest Trade-off Without Any Retraining

A training-free inference-time technique for diffusion super-resolution that separates images into frequency bands and iteratively refines them to beat the perception-fidelity trade-off.

Pro

Paper, Full Access
  • IAFS is a training-free inference-time method for diffusion super-resolution that beats the perception-fidelity trade-off.
  • It splits each particle into high/low-frequency parts and fuses low-frequency info from similar neighbors via AdaIN.
  • Different reward mixes (CLIPIQA and LPIPS) are used at different timesteps based on frequency behavior.
  • Iterative refinement uses each round's output as pseudo-GT, with n=3 iterations offering the best balance.
  • Tested on ResShift, DiffBIR, and SeeSR across ImageNet, DIV2K, RealSR, DRealSR.
  • Paper reports gains on both structural (PSNR/SSIM) and perceptual (LPIPS/CLIPIQA/MUSIQ/MANIQA) metrics simultaneously.

Diffusion-based super-resolution has a well-known split personality. Push it toward crisp textures and edges look invented. Push it toward faithful reconstruction and everything turns plasticky. A new training-free method called Iterative Inference-Time Scaling with Adaptive Frequency Steering (IAFS) attacks this trade-off by treating the problem as one of frequency allocation, and it improves both sides of the ledger at once across three popular diffusion SR backbones.

Existing inference-time scaling tricks each pick a poison. Particle optimization methods often produce poor perceptual quality, while optimal-path search methods lose structural fidelity. IAFS is designed to sit between them.

Diagram comparing particle optimization, optimal path search, and the proposed IAFS approach across diffusion timesteps

Frequencies beat rewards as the right handle

The authors work out something clean in the Fourier domain: because noise variance in the reverse process shrinks over time, the amplitude spectrum of the intermediate latent decays as a Gaussian in frequency magnitude. In practice, low-frequency structure is largely fixed early in the reverse diffusion, while high-frequency detail is only recovered in the final steps. They also lean on a useful complementarity between two common metrics: CLIPIQA is sensitive to high-frequency content, while LPIPS favors mid-to-low-frequency structure.