RaySplats: Ray Tracing based Gaussian Splatting

TL;DR

RaySplats introduces a ray-tracing based Gaussian Splatting method that enhances 3D object creation from 2D images by enabling realistic lighting and shadow effects, overcoming limitations of rasterization-based approaches.

Contribution

The paper presents RaySplats, a novel approach that integrates ray tracing with Gaussian Splatting, allowing for realistic lighting and shadow rendering in 3D reconstructions.

Findings

Enables incorporation of lights and shadows in Gaussian Splatting

Uses ray-ellipse intersection for rendering

Improves realism in 3D object creation

Abstract

3D Gaussian Splatting (3DGS) is a process that enables the direct creation of 3D objects from 2D images. This representation offers numerous advantages, including rapid training and rendering. However, a significant limitation of 3DGS is the challenge of incorporating light and shadow reflections, primarily due to the utilization of rasterization rather than ray tracing for rendering. This paper introduces RaySplats, a model that employs ray-tracing based Gaussian Splatting. Rather than utilizing the projection of Gaussians, our method employs a ray-tracing mechanism, operating directly on Gaussian primitives represented by confidence ellipses with RGB colors. In practice, we compute the intersection between ellipses and rays to construct ray-tracing algorithms, facilitating the incorporation of meshes with Gaussian Splatting models and the addition of lights, shadows, and other related effects.