Yonsei University's DyMoS Fixes Frozen AI Videos With One Slider
DyMoS fixes the root cause of static I2V video generation by rebalancing self-attention away from the reference frame, no retraining required
- Root cause found: I2V models produce static videos because generated frames over-attend to the reference frame's tokens, suppressing inter-frame motion.
- DyMoS fix: A training-free, model-agnostic method that rebalances self-attention away from the reference frame only during early denoising steps.
- Single scalar control: One parameter continuously controls motion strength from static to highly dynamic, with no model retraining or image modification.
- Benchmark results: +3.4% VBench score improvement; outperforms baselines on motion, fidelity, and overall quality simultaneously on CogVideoX and Wan 2.2.
- Plug-and-play: Works across multiple state-of-the-art I2V backbones with no architectural changes; code is open-source on GitHub.
- Limitation: Can only unlock motion the base model is capable of; does not help with complex multi-body interactions the backbone never learned.
Image-to-video (I2V) models have a stubborn problem: give them a stunning reference image and ask them to animate it, and the result is often a video that barely moves. Researchers from Yonsei University and Adobe Research have now pinpointed exactly why this happens and built a clean, training-free fix called DyMoS (Dynamic Motion Slider).
Recent text-to-video models have demonstrated strong capabilities in producing high-quality, dynamic videos, but when those same models are fine-tuned to support image-to-video generation, the adaptation frequently suppresses motion dynamics, resulting in more static outputs compared to their text-to-video counterparts. The gap is not a quirk of any single model -- it is a systemic pattern across the field.
The attention trap hiding in plain sight
To understand the fix, you need to understand the mechanism. Modern I2V diffusion models work by treating the reference image as an extra frame that participates in the model's self-attention layers alongside the frames being generated. Self-attention is the mechanism that lets each token (a small patch of a frame) look at all other tokens and decide what information to pull in.
The paper identifies reference-frame dominance as a key mechanism behind motion suppression: non-reference frames in I2V models allocate excessive self-attention to reference-frame key tokens, causing reference information to be over-propagated across time and suppressing inter-frame dynamics. In plain terms, every generated frame is spending too much of its "attention budget" staring at the reference image instead of talking to its neighboring frames. The result is a video where every frame looks like a slightly noisy copy of the input -- technically coherent, but visually frozen.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
