Peking University's BVE Edits 3D Assets From Text Without Manual Masking
A cross-institution team introduces BVE, a text-guided 3D editing framework that adds trainable modules onto an image-to-3D backbone and uses automatic 3D masks to keep unchanged regions intact.
Takeaways
- Researchers from Peking University, HKUST(GZ), and Microsoft propose BVE, a text-guided 3D editing framework.
- Built as lightweight trainable modules on top of a frozen image-to-3D backbone like TRELLIS.
- Introduces an annotation-free 3D masking strategy so unchanged regions stay intact automatically.
- Supports global add operations and local replace or restyle edits in seconds.
- Ships with a self-constructed dataset pipeline using SAM2, Florence-2, Gemma3, and Flux/Qwen.
- Outperforms Tailor3D, Vox-e, TRELLIS, and Hunyuan qualitatively across six edit prompt types.
Editing a 3D asset the way you edit an image, by typing a prompt and having only the part you named change, is still surprisingly hard. A new paper titled Beyond Voxel 3D Editing, from authors at Peking University, HKUST(GZ), Microsoft AI, and Microsoft Research, proposes a framework called BVE that supports global edits like adding an object alongside local edits like replacement or style changes in seconds, producing detailed meshes for geometry and 3D Gaussians or radiance fields for appearance.
Why 3D editing keeps breaking
Two families of methods dominate today, and both bleed quality. Multi-view approaches incur losses when projecting back to 3D, while voxel-based editing is constrained in both the regions that can be modified and the scale of modifications. Compounding this, the lack of sufficiently large editing datasets for training and evaluation remains a challenge, which is why most published demos stick to a handful of hand-picked objects.
Context matters here. The backbone the authors build on is TRELLIS, a large image-to-3D model that uses a unified Structured LATent (SLAT) representation with rectified flow transformers, pretrained on 500K diverse 3D assets, with flexible output formats and some local 3D editing capabilities. TRELLIS supports resampling voxel regions, but as one follow-up paper on the same backbone points out, edits are applied by region-wise resampling rather than replacement, with no constraints on scale or location. In practice, ask it to swap a chair's cushion and you can also lose the frame.