HybridSplat: Fast Reflection-baked Gaussian Tracing using Hybrid Splatting

TL;DR

HybridSplat introduces a hybrid Gaussian splatting method with reflection baking and acceleration techniques, significantly improving rendering speed and reducing memory usage for complex reflective scenes in novel view synthesis.

Contribution

It proposes a novel reflection-baked Gaussian tracing mechanism integrated into a hybrid splatting framework, enhancing speed and efficiency in scene rendering.

Findings

7x faster rendering speed on complex scenes

4x fewer primitives needed compared to baselines

State-of-the-art performance in reflective scene rendering

Abstract

Rendering complex reflection of real-world scenes using 3D Gaussian splatting has been a quite promising solution for photorealistic novel view synthesis, but still faces bottlenecks especially in rendering speed and memory storage. This paper proposes a new Hybrid Splatting(HybridSplat) mechanism for Gaussian primitives. Our key idea is a new reflection-baked Gaussian tracing, which bakes the view-dependent reflection within each Gaussian primitive while rendering the reflection using tile-based Gaussian splatting. Then we integrate the reflective Gaussian primitives with base Gaussian primitives using a unified hybrid splatting framework for high-fidelity scene reconstruction. Moreover, we further introduce a pipeline-level acceleration for the hybrid splatting, and reflection-sensitive Gaussian pruning to reduce the model size, thus achieving much faster rendering speed and lower memory storage while preserving the reflection rendering quality. By extensive evaluation, our HybridSplat accelerates about 7x rendering speed across complex reflective scenes from Ref-NeRF, NeRF-Casting with 4x fewer Gaussian primitives than similar ray-tracing based Gaussian splatting baselines, serving as a new state-of-the-art method especially for complex reflective scenes.