TU Munich's GS4City Teaches 3D City Models to Recognize Windows and Walls
A new method fuses CityGML building priors with 3D Gaussian Splatting to deliver hierarchical, geometrically clean urban scene understanding.
ProTakeaways
- GS4City injects hierarchical CityGML LoD3 priors into 3D Gaussian Splatting scenes for urban semantics.
- Two-pass raycasting recovers facade windows and doors that single-pass projection loses behind wall geometry.
- Fuses geometry-grounded masks with foundation-model predictions, then learns per-Gaussian identity codes with 2D and 3D losses.
- Beats LangSplat and Gaga by up to 15.8 IoU points on coarse building segmentation across TUM2TWIN and Gold Coast.
- Fine-grained mIoU reaches 44.4 percent on Gold Coast versus 30.2 for LangSplat and 23.5 for Gaga.
- Code available at github.com/Jinyzzz/GS4City; still needs aligned LoD3 city models to work.
Photorealistic city reconstructions built from 3D Gaussian Splatting look beautiful, but they are semantically dumb. Ask them where the windows end and the walls begin, and you get fuzzy blobs. A team from TU Munich, KIT, and the University of Cambridge is proposing a fix by borrowing something urban planners have had for years: structured city models with hand-labeled facades, roofs, and doors. Their method, GS4City, plugs those priors directly into the Gaussian scene and posts double-digit gains over popular semantic 3DGS baselines.
Why 2D foundation models keep tripping on buildings
Most semantic 3DGS pipelines lift labels from 2D models like SAM or CLIP into the Gaussian field. That approach works for generic object discovery but breaks on cities, often yielding ambiguous boundaries and limited support for structured urban semantics. Facades have repeated patterns, reflective glass, and heavy occlusions, so windows get absorbed into walls and roofs bleed into sky.
CityGML, the OGC standard for 3D city models, already encodes what these pipelines are missing. At Level of Detail 3 (LoD3), it carries georeferenced polygonal surfaces with explicit parent-child relations, meaning a Window belongs to a WallSurface, which belongs to a Building. Directly transferring those 3D labels to Gaussian primitives, however, is unreliable. City models define semantics on explicit surfaces, whereas 3DGS relies on alpha-composed primitives that lack strict surface ownership. Floaters and occlusions in urban scenes aggravate this mismatch further, making 3D assignment highly unstable near building boundaries.