Neo: Real-Time On-Device 3D Gaussian Splatting with Reuse-and-Update Sorting Acceleration

TL;DR

Neo enables real-time 3D Gaussian Splatting on resource-limited devices by reusing and updating sorting, significantly boosting throughput and reducing memory bandwidth for immersive AR/VR experiences.

Contribution

It introduces a reuse-and-update sorting algorithm and a hardware accelerator, optimizing 3D Gaussian Splatting for real-time on-device rendering.

Findings

Achieves up to 10.0x throughput improvement over state-of-the-art edge GPU.

Reduces DRAM traffic by over 94%.

Enables practical high-quality, low-latency 3D rendering on resource-constrained devices.

Abstract

3D Gaussian Splatting (3DGS) rendering in real-time on resource-constrained devices is essential for delivering immersive augmented and virtual reality (AR/VR) experiences. However, existing solutions struggle to achieve high frame rates, especially for high-resolution rendering. Our analysis identifies the sorting stage in the 3DGS rendering pipeline as the major bottleneck due to its high memory bandwidth demand. This paper presents Neo, which introduces a reuse-and-update sorting algorithm that exploits temporal redundancy in Gaussian ordering across consecutive frames, and devises a hardware accelerator optimized for this algorithm. By efficiently tracking and updating Gaussian depth ordering instead of re-sorting from scratch, Neo significantly reduces redundant computations and memory bandwidth pressure. Experimental results show that Neo achieves up to 10.0x and 5.6x higher throughput than state-of-the-art edge GPU and ASIC solution, respectively, while reducing DRAM traffic by 94.5% and 81.3%. These improvements make high-quality and low-latency on-device 3D rendering more practical.