Vorion: A RISC-V GPU with Hardware-Accelerated 3D Gaussian Rendering and Training

TL;DR

Vorion introduces the first GPGPU prototype with hardware-accelerated 3D Gaussian rendering and training, enabling real-time neural rendering and 3D scene generation on edge devices and workstations.

Contribution

It presents a scalable hardware architecture for 3D Gaussian Splatting, with minimal modifications to traditional rasterizers, and demonstrates prototype implementation and performance results.

Findings

Achieves 19 FPS for rendering on prototype hardware.

Reaches 38.6 training iterations per second with scaled design.

Demonstrates feasibility of real-time 3D Gaussian rendering and training.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a foundational technique for real-time neural rendering, 3D scene generation, volumetric video (4D) capture. However, its rendering and training impose massive computation, making real-time rendering on edge devices and real-time 4D reconstruction on workstations currently infeasible. Given its fixed-function nature and similarity with traditional rasterization, 3DGS presents a strong case for dedicated hardware in the graphics pipeline of next-generation GPUs. This work, Vorion, presents the first GPGPU prototype with hardware-accelerated 3DGS rendering and training. Vorion features scalable architecture, minimal hardware change to traditional rasterizers, z-tiling to increase parallelism, and Gaussian/pixel-centric hybrid dataflow. We prototype the minimal system (8 SIMT cores, 2 Gaussian rasterizer) using TSMC 16nm FinFET technology, which achieves 19 FPS for rendering. The scaled design with 16 rasterizers achieves 38.6 iterations/s for training.