Skip to main content
Back

CPAM Cuts Background Distortion in Half Without Fine-Tuning Stable Diffusion

Researchers from University of Science (VNU-HCM), Monash, and Dayton unveil a training-free image editor that preserves backgrounds while modifying non-rigid objects.

Pro

  • CPAM is a zero-shot image editing framework from University of Science (VNU-HCM), Monash, and University of Dayton, accepted to IEEE TMM 2026.
  • It rewrites self-attention and cross-attention to edit objects while keeping backgrounds pixel-stable, no fine-tuning needed.
  • Works out of the box with Stable Diffusion 1.5, 2.1, and SDXL through mask-guided attention manipulation.
  • New IMBA benchmark with 104 samples for object retention, modification, and background alteration.
  • CPAM-SDXL cuts background LPIPS to 0.118 versus 0.182 for the best prior baseline, DiffEdit.
  • Code and dataset are open-source; Gradio demo announced as coming soon.

A joint team from the University of Science (VNU-HCM), Monash University, and the University of Dayton has released CPAM, a zero-shot framework for one of the messiest problems in diffusion-based image editing: changing a specific object in a real photo without warping the rest of the scene. The work, accepted to IEEE Transactions on Multimedia, plugs into Stable Diffusion 1.5, 2.1, and SDXL with no fine-tuning.

When the background refuses to sit still

Text-guided editors like SDEdit, MasaCtrl, and Imagic tend to fail in predictable ways. They smear textures, lose object identity, demand extensive fine-tuning, and struggle to confine edits to a specific region. Ask them to change a dog's pose or swap a cup for a mug, and the wallpaper shifts, the lighting drifts, or the subject melts into something new.

The root cause lives in how diffusion models handle attention. Self-attention leaks features across the whole image, so preserving a background means fighting the model. Cross-attention lets a target prompt bleed into pixels it was never meant to touch. CPAM (Context-Preserving Adaptive Manipulation) attacks both failure modes directly.