CityGo: Lightweight Urban Modeling and Rendering with Proxy Buildings and Residual Gaussians

TL;DR

CityGo introduces a hybrid urban scene modeling framework combining proxy geometry and residual Gaussians, enabling real-time, photorealistic rendering of large-scale cities on mobile devices with reduced training time and memory.

Contribution

The paper presents a novel hybrid approach that integrates textured proxy meshes with residual and surrounding Gaussians for efficient, high-fidelity urban scene rendering from aerial views.

Findings

Achieves 1.4x faster training compared to pure Gaussian Splatting.

Enables real-time rendering on mobile GPUs with lower memory and energy use.

Maintains high visual quality comparable to existing Gaussian-based methods.

Abstract

Accurate and efficient modeling of large-scale urban scenes is critical for applications such as AR navigation, UAV based inspection, and smart city digital twins. While aerial imagery offers broad coverage and complements limitations of ground-based data, reconstructing city-scale environments from such views remains challenging due to occlusions, incomplete geometry, and high memory demands. Recent advances like 3D Gaussian Splatting (3DGS) improve scalability and visual quality but remain limited by dense primitive usage, long training times, and poor suit ability for edge devices. We propose CityGo, a hybrid framework that combines textured proxy geometry with residual and surrounding 3D Gaussians for lightweight, photorealistic rendering of urban scenes from aerial perspectives. Our approach first extracts compact building proxy meshes from MVS point clouds, then uses zero order SH Gaussians to generate occlusion-free textures via image-based rendering and back-projection. To capture high-frequency details, we introduce residual Gaussians placed based on proxy-photo discrepancies and guided by depth priors. Broader urban context is represented by surrounding Gaussians, with importance-aware downsampling applied to non-critical regions to reduce redundancy. A tailored optimization strategy jointly refines proxy textures and Gaussian parameters, enabling real-time rendering of complex urban scenes on mobile GPUs with significantly reduced training and memory requirements. Extensive experiments on real-world aerial datasets demonstrate that our hybrid representation significantly reduces training time, achieving on average 1.4x speedup, while delivering comparable visual fidelity to pure 3D Gaussian Splatting approaches. Furthermore, CityGo enables real-time rendering of large-scale urban scenes on mobile consumer GPUs, with substantially reduced memory usage and energy consumption.