Unified Gaussian Primitives for Scene Representation and Rendering

TL;DR

This paper introduces a unified scene representation using 3D Gaussian primitives that effectively model diverse appearance types, support global illumination, and are compatible with differentiable rendering, addressing limitations of traditional mesh and voxel methods.

Contribution

The paper presents a novel 3D Gaussian primitive-based scene representation that unifies appearance modeling, enables physically-based scattering, and supports efficient differentiable rendering.

Findings

Versatile rendering of glossy to fuzzy elements

Supports global illumination and appearance editing

Efficient reproduction of details compared to volumetric methods

Abstract

Searching for a unified scene representation remains a research challenge in computer graphics. Traditional mesh-based representations are unsuitable for dense, fuzzy elements, and introduce additional complexity for filtering and differentiable rendering. Conversely, voxel-based representations struggle to model hard surfaces and suffer from intensive memory requirement. We propose a general-purpose rendering primitive based on 3D Gaussian distribution for unified scene representation, featuring versatile appearance ranging from glossy surfaces to fuzzy elements, as well as physically based scattering to enable accurate global illumination. We formulate the rendering theory for the primitive based on non-exponential transport and derive efficient rendering operations to be compatible with Monte Carlo path tracing. The new representation can be converted from different sources, including meshes and 3D Gaussian splatting, and further refined via transmittance optimization thanks to its differentiability. We demonstrate the versatility of our representation in various rendering applications such as global illumination and appearance editing, while supporting arbitrary lighting conditions by nature. Additionally, we compare our representation to existing volumetric representations, highlighting its efficiency to reproduce details.