SLTarch: Towards Scalable Point-Based Neural Rendering by Taming Workload Imbalance and Memory Irregularity

TL;DR

SLTarch is a co-designed algorithm-architecture framework that significantly improves the efficiency and energy consumption of point-based neural rendering, enabling real-time performance on mobile platforms.

Contribution

It introduces SLTree, LTcore, and SPcore, novel data structures and hardware components that address workload imbalance and memory irregularity in PBNR.

Findings

3.9× speedup over mobile GPU

98% energy savings on mobile platforms

1.8× speedup over existing accelerators

Abstract

Rendering is critical in fields like 3D modeling, AR/VR, and autonomous driving, where high-quality, real-time output is essential. Point-based neural rendering (PBNR) offers a photorealistic and efficient alternative to conventional methods, yet it is still challenging to achieve real-time rendering on mobile platforms. We pinpoint two major bottlenecks in PBNR pipelines: LoD search and splatting. LoD search suffers from workload imbalance and irregular memory access, making it inefficient on off-the-shelf GPUs. Meanwhile, splatting introduces severe warp divergence across GPU threads due to its inherent sparsity. To tackle these challenges, we propose SLTarch, an algorithm-architecture co-designed framework. At its core, SLTarch introduces SLTree, a dedicated subtree-based data structure, and LTcore, a specialized hardware architecture tailored for efficient LoD search. Additionally, we co-design a divergence-free splatting algorithm with our simple yet principled hardware augmentation, SPcore, to existing PBNR accelerators. Compared to a mobile GPU, SLTarch achieves 3.9$\times$ speedup and 98\% energy savings with negligible architecture overhead. Compared to existing accelerator designs, SLTarch achieves 1.8$\times$ speedup with 54\% energy savings.