MIT CSAIL's Creo Fixes AI Image Drift With Staged Editable Generation
MIT researchers propose a staged text-to-image workflow with sketch-level intermediates and decision locking, boosting user control and output diversity.
PRO- MIT CSAIL introduces Creo, a staged text-to-image system that replaces one-shot generation with editable sketch-to-render pipeline.
- Pipeline moves through viewpoint, composition, color, lighting, and style stages, exposing intermediate abstractions for user edits.
- Locking mechanism preserves prior decisions; system applies diffs instead of regenerating full images, reducing edit drift.
- Users can draw, erase, mask regions, and describe localized changes without affecting unrelated elements of the image.
- Study participants reported stronger ownership over outputs and could trace their decisions across generation stages.
- Embedding-based analysis shows Creo outputs are measurably less homogeneous than one-shot text-to-image results.
Text-to-image models have gotten spectacular at producing polished pictures from a single prompt, but they treat image-making like a one-shot vending machine. You type, they decide, and if you want to change the color of a jacket, half the composition mysteriously shifts too. A team at MIT CSAIL argues this is misaligned with how visual ideas actually form, and they built a system called Creo to prove there is a better way.
The paper, from Zoe De Simone, Angie Boggust, Fredo Durand, Ashia Wilson, and Arvind Satyanarayan, reframes T2I generation as a staged, editable pipeline rather than a black-box render. Rather than jumping straight to a high-fidelity image, Creo walks users through progressively detailed abstractions, letting them lock in decisions along the way.
Why one-shot generation gets in your way
The authors identify three concrete failure modes of current T2I systems. Models make implicit visual decisions on behalf of the user, introduce fine-grained details that anchor users prematurely and limit their ability to keep options open early on, and cause unintended changes during editing that are difficult to correct.
That last problem, often called drift, is familiar to anyone who has tried to iterate with a diffusion model. Small prompt tweaks can rewrite the entire scene, because each generation is essentially a fresh sample from the model rather than an edit to the previous one. Premature anchoring is subtler but equally damaging for ideation: once you see a specific rendered face, it is hard to imagine any other face for that character.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
