LaviGen Places 3D Furniture 97% Collision-Free and 6x Faster
A new framework treats 3D scene layout as an autoregressive geometry problem, hitting 19% better physical plausibility and 65% faster inference than the prior state of the art.
Takeaways
- LaviGen repurposes 3D diffusion models for layout generation, working directly in native 3D voxel space instead of via text
- Uses autoregressive placement: encodes scene plus object, denoises to get updated scene, repeat for each item
- Hits 97.3 Collision-Free and 98.6 In-Boundary scores on the LayoutVLM benchmark, 19% above prior SOTA
- Dual-guidance self-rollout distillation fixes exposure bias and cuts inference to 24 seconds, a 65% speedup
- Same framework supports layout completion and editing (insert, remove, replace) without retraining
- Code available at github.com/fenghora/LaviGen, builds on TRELLIS with a 3B DiT
Arranging furniture in a virtual room sounds trivial until you ask a model to do it. Chairs float above the floor. Tables clip through walls. Sofas overlap. Most current text-to-3D layout systems try to solve this by having a large language model spit out JSON with object coordinates, which produces semantically reasonable but physically broken scenes. A new paper called LaviGen takes a different route: it repurposes a pretrained 3D asset generator to place objects directly in 3D space, treating the whole thing as an autoregressive sequence of geometry updates.

Why layout-as-language keeps breaking
The dominant paradigm for 3D scene layout has been to prompt an LLM with a room description and have it output object positions as structured text. LayoutGPT does this, and follow-ups like Holodeck and I-Design layer on iterative refinement. The problem is fundamental. Without physical modeling, spatial layouts end up inconsistent, producing object collisions, inter-penetrations, or floating geometry. A language model has no built-in notion of what it means for two bounding boxes to intersect.
The prior state of the art, LayoutVLM, tried to patch this by rendering the layout to 2D images and using a vision-language model plus differentiable optimization to nudge poses. It works better, but image-level supervision is computationally costly and lacks a fundamental understanding of 3D spatial structure. On cluttered scenes with many objects, its inference time balloons.
Geometry priors were hiding in plain sight
The authors noticed something obvious in hindsight. Modern 3D generative models like TRELLIS have already learned rich priors about how objects occupy 3D space, because they were trained on hundreds of thousands of assets. A scene layout is just another kind of geometric distribution. So instead of asking an LLM to reason about coordinates, why not ask a 3D diffusion model to generate the scene's voxel occupancy directly, one object at a time?