ByteDance's CAFM Cuts Image Generation Error in Half Without Extra Guidance
ByteDance Seed's continuous adversarial flow models replace mean squared error with a learned critic, cutting ImageNet guidance-free FID by more than half.
ProTakeaways
- ByteDance Seed's CAFM replaces flow matching's MSE loss with a learned adversarial discriminator.
- ImageNet 256px guidance-free FID improves from 8.26 to 3.63 on SiT, 7.17 to 3.57 on JiT.
- Guided FID also improves: SiT 2.06 to 1.53, JiT 1.86 to 1.80, with only 10 fine-tuning epochs.
- Discriminator operates in JVP derivative space, extending adversarial training to continuous time cleanly.
- Text-to-image gains on Z-Image: GenEval 0.81 to 0.85, DPG-Bench 83.7 to 85.2.
- Primarily a post-training recipe; from-scratch training works but converges slower than plain flow matching.
Flow matching has quietly become the default engine behind modern image and video generators, but it has a stubborn habit of producing off-manifold samples unless you crank up classifier-free guidance. A team from ByteDance Seed proposes trading the fixed mean-squared-error objective for a learned adversarial critic, and the numbers on ImageNet are hard to ignore: guidance-free FID drops from 8.26 to 3.63 on latent-space SiT and from 7.17 to 3.57 on pixel-space JiT, using just 10 epochs of fine-tuning.
The method, called Continuous Adversarial Flow Models (CAFMs), keeps everything about a flow-matching model the same but post-trains it with a GAN-style discriminator instead of MSE. The result is a drop-in objective swap that improves both guidance-free and guided generation.
Why MSE is quietly hurting your samples
Flow matching learns the velocity field of a probability flow between noise and data distributions by minimizing the squared L2 loss between predicted and target velocities. With infinite capacity, this converges to the ground-truth flow. In practice, finite-capacity networks have to generalize, and the L2 loss measures Euclidean distance rather than distance along the data manifold. That induces incorrect generalization relative to the underlying data distribution.
That is why models often need classifier-free guidance to look sharp, and why guidance itself is a bit of a hack: it improves quality by biasing sampling away from the true learned distribution. Prior fixes each had trade-offs:
- Riemannian flow matching extends flow matching to non-Euclidean geometries, but requires manual definition of the data manifold, which is often unknown for general datasets.