VR-Pipe: Streamlining Hardware Graphics Pipeline for Volume Rendering

TL;DR

VR-Pipe enhances hardware graphics pipelines for volume rendering, specifically radiance fields, by integrating native support for early termination and multi-granular tile binning, resulting in significant speedups with minimal hardware overhead.

Contribution

It introduces two hardware innovations—early termination support and tile binning—that optimize volume rendering performance in modern GPUs.

Findings

Achieves up to 2.78x speedup in rendering performance.

Seamlessly integrates into existing graphics hardware with negligible overhead.

Effectively supports state-of-the-art radiance field methods like 3D Gaussian splatting.

Abstract

Graphics rendering that builds on machine learning and radiance fields is gaining significant attention due to its outstanding quality and speed in generating photorealistic images from novel viewpoints. However, prior work has primarily focused on evaluating its performance through software-based rendering on programmable shader cores, leaving its performance when exploiting fixed-function graphics units largely unexplored. In this paper, we investigate the performance implications of performing radiance field rendering on the hardware graphics pipeline. In doing so, we implement the state-of-the-art radiance field method, 3D Gaussian splatting, using graphics APIs and evaluate it across synthetic and real-world scenes on today's graphics hardware. Based on our analysis, we present VR-Pipe, which seamlessly integrates two innovations into graphics hardware to streamline the hardware pipeline for volume rendering, such as radiance field methods. First, we introduce native hardware support for early termination by repurposing existing special-purpose hardware in modern GPUs. Second, we propose multi-granular tile binning with quad merging, which opportunistically blends fragments in shader cores before passing them to fixed-function blending units. Our evaluation shows that VR-Pipe greatly improves rendering performance, achieving up to a 2.78x speedup over the conventional graphics pipeline with negligible hardware overhead.