Mobile-GS: Real-time Gaussian Splatting for Mobile Devices

TL;DR

Mobile-GS introduces a real-time, efficient Gaussian Splatting method optimized for mobile devices, combining order-independent rendering, view-dependent effects, and model compression to enable high-quality 3D rendering on edge hardware.

Contribution

It proposes a novel mobile-tailored Gaussian Splatting approach with order-independent rendering, neural view-dependent enhancement, and model compression techniques for real-time mobile deployment.

Findings

Achieves real-time rendering on mobile devices.

Reduces model size with neural vector quantization.

Maintains high visual quality in mobile scenarios.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful representation for high-quality rendering across a wide range of applications.However, its high computational demands and large storage costs pose significant challenges for deployment on mobile devices. In this work, we propose a mobile-tailored real-time Gaussian Splatting method, dubbed Mobile-GS, enabling efficient inference of Gaussian Splatting on edge devices. Specifically, we first identify alpha blending as the primary computational bottleneck, since it relies on the time-consuming Gaussian depth sorting process. To solve this issue, we propose a depth-aware order-independent rendering scheme that eliminates the need for sorting, thereby substantially accelerating rendering. Although this order-independent rendering improves rendering speed, it may introduce transparency artifacts in regions with overlapping geometry due to the scarcity of rendering order. To address this problem, we propose a neural view-dependent enhancement strategy, enabling more accurate modeling of view-dependent effects conditioned on viewing direction, 3D Gaussian geometry, and appearance attributes. In this way, Mobile-GS can achieve both high-quality and real-time rendering. Furthermore, to facilitate deployment on memory-constrained mobile platforms, we also introduce first-order spherical harmonics distillation, a neural vector quantization technique, and a contribution-based pruning strategy to reduce the number of Gaussian primitives and compress the 3D Gaussian representation with the assistance of neural networks. Extensive experiments demonstrate that our proposed Mobile-GS achieves real-time rendering and compact model size while preserving high visual quality, making it well-suited for mobile applications.