UrbanGS: Semantic-Guided Gaussian Splatting for Urban Scene Reconstruction

TL;DR

UrbanGS introduces a semantic-guided approach to urban scene reconstruction that effectively distinguishes static and dynamic objects, improving accuracy and efficiency without manual annotations.

Contribution

The paper presents UrbanGS, a novel method that leverages 2D semantic maps and temporal embeddings to separately process static and dynamic scene elements, enhancing reconstruction quality.

Findings

Outperforms state-of-the-art methods in reconstruction quality

Accurately preserves static content while capturing dynamic elements

Improves efficiency by avoiding unnecessary updates for static objects

Abstract

Reconstructing urban scenes is challenging due to their complex geometries and the presence of potentially dynamic objects. 3D Gaussian Splatting (3DGS)-based methods have shown strong performance, but existing approaches often incorporate manual 3D annotations to improve dynamic object modeling, which is impractical due to high labeling costs. Some methods leverage 4D Gaussian Splatting (4DGS) to represent the entire scene, but they treat static and dynamic objects uniformly, leading to unnecessary updates for static elements and ultimately degrading reconstruction quality. To address these issues, we propose UrbanGS, which leverages 2D semantic maps and an existing dynamic Gaussian approach to distinguish static objects from the scene, enabling separate processing of definite static and potentially dynamic elements. Specifically, for definite static regions, we enforce global consistency to prevent unintended changes in dynamic Gaussian and introduce a K-nearest neighbor (KNN)-based regularization to improve local coherence on low-textured ground surfaces. Notably, for potentially dynamic objects, we aggregate temporal information using learnable time embeddings, allowing each Gaussian to model deformations over time. Extensive experiments on real-world datasets demonstrate that our approach outperforms state-of-the-art methods in reconstruction quality and efficiency, accurately preserving static content while capturing dynamic elements.