GrOCE Erases AI Image Concepts in 1.73 Seconds Without Breaking Neighbors
A training-free framework uses semantic graphs to precisely erase concepts from diffusion models in under two seconds, without retraining or collateral damage.
ProTakeaways
- GrOCE is a training-free concept erasure method for text-to-image diffusion models using dynamic semantic graphs.
- Three stages: build graph over token embeddings, identify target cluster via graph diffusion, sever high-influence tokens from prompt.
- Erases 10 concepts in 1.73 seconds versus 3.59 for AdaVD and over 8,500 for ConAbl.
- Achieves FID of 0 on non-target concepts across cartoon characters and art styles like Van Gogh and Picasso.
- Works across SD 1.4, 1.5, 2.1, 3.0, SDXL 1.0, and FLUX.1-schnell without modification.
- Ablation shows the Identify stage is critical: removing it drops FID from 0 to 426.74 through over-erasure.
Removing a specific concept from a text-to-image diffusion model sounds simple until you try it. You want to strip out Van Gogh's style, or block generations of a copyrighted cartoon character, without breaking the model's understanding of impressionism, brushstrokes, or every bear-adjacent animal. Most existing approaches either fine-tune the entire model at high cost or apply crude activation edits that leak into unrelated concepts. A new paper introduces GrOCE (Graph-Guided Online Concept Erasure), a training-free method that reframes the problem as a graph cut over semantic relationships, and reports both perfect preservation of non-target content and an order-of-magnitude speedup over prior work.
Why erasure keeps breaking neighboring concepts
Unlike discriminative models with discrete decision boundaries, diffusion models distribute concept representations across temporal denoising trajectories, spatial attention maps, and cross-modal embeddings simultaneously. That entanglement is the core issue. Modifying one concept like violence can unintentionally alter semantically adjacent or visually similar concepts such as conflict, action, or even intensity.
The authors group prior work into two flawed camps. Parameter-based methods like ESD and CA align the probability distributions of the targeted concept with that of a null string, but this incurs high computational cost, suffers from catastrophic forgetting, and struggles to adapt to emerging risks. Newer adapter-based approaches like MACE and SPM cut the forgetting problem but still need substantial fine-tuning time. Inference-time methods such as UCE, SPEED, and AdaVD skip retraining, but rely on heuristic mappings that miss deeper semantic entanglements, making accurate boundary detection elusive. Erase "Bear" with one of these and you often still get grizzlies and polar bears.