Mira-Scene Rebuilds Editable 3D Scenes From a Single Photo
A new research paper proposes Canonical Coordinate Maps, a dense pixel-aligned representation that reconstructs full 3D scenes from a single image.
- Researchers introduce Mira-Scene, a single-image compositional 3D scene reconstruction framework.
- Core idea: Canonical Coordinate Map (CCM), a dense pixel-aligned field replacing sparse pose regression.
- CCM + monocular Point Cloud Map yields correspondences solved via Umeyama geometric alignment.
- Uses a multimodal diffusion transformer with separate geometry and layout expert streams.
- Beats SAM3D by 39.8% relative 3D-IoU and 16.5% 2D-IoU on layout accuracy.
- Code and checkpoint released on GitHub and Hugging Face.
Researchers led by Yang-Tian Sun describe Mira-Scene, a framework that reconstructs an editable 3D scene from a single image by generating separate object meshes and estimating each object’s position, orientation, and scale. In their paper, the authors report relative gains of 39.8% in 3D intersection over union and 16.5% in 2D intersection over union against SAM3D. The framework predicts dense pixel correspondences, then recovers object transforms through geometric alignment.
Why scene layout remains difficult
Single-image generators can produce detailed 3D objects, but assembling those assets into a coherent room, street, or diorama requires consistent depth, orientation, scale, and contact between objects. Small placement errors accumulate quickly: a plausible chair model still fails as part of a scene when it floats above the floor, intersects a table, or has the wrong scale.
| Approach | Strength | Tradeoff |
|---|---|---|
| Holistic generation | Models the full scene jointly | Often reduces object-level detail and editability |
| Compositional generation | Preserves separate, editable objects | Must infer a reliable transform for every object |
Training data compounds the problem. Object-level 3D datasets provide abundant geometry, while scene datasets with labeled per-object poses remain limited. Directly regressing a small set of pose values from scarce scene supervision can generalize poorly across object categories and environments. Mira-Scene converts layout estimation into a dense prediction task that can use object-level 3D data without requiring scene-level pose annotations.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
