From Volume Rendering to 3D Gaussian Splatting: Theory and Applications

TL;DR

This paper reviews 3D Gaussian Splatting (3DGS), a scene representation technique that enables efficient real-time rendering and applications in 3D reconstruction, surface modeling, and content creation, while discussing its limitations and solutions.

Contribution

It provides a comprehensive overview of 3DGS, including its formulation, challenges, and diverse applications, highlighting recent advances and future directions.

Findings

3DGS enables real-time rendering and novel view synthesis.

It effectively supports surface reconstruction and avatar modeling.

Limitations include high memory usage and limited secondary-ray effects support.

Abstract

The problem of 3D reconstruction from posed images is undergoing a fundamental transformation, driven by continuous advances in 3D Gaussian Splatting (3DGS). By modeling scenes explicitly as collections of 3D Gaussians, 3DGS enables efficient rasterization through volumetric splatting, offering thus a seamless integration with common graphics pipelines. Despite its real-time rendering capabilities for novel view synthesis, 3DGS suffers from a high memory footprint, the tendency to bake lighting effects directly into its representation, and limited support for secondary-ray effects. This tutorial provides a concise yet comprehensive overview of the 3DGS pipeline, starting from its splatting formulation and then exploring the main efforts in addressing its limitations. Finally, we survey a range of applications that leverage 3DGS for surface reconstruction, avatar modeling, animation, and content generation-highlighting its efficient rendering and suitability for feed-forward pipelines.