3DGabSplat: 3D Gabor Splatting for Frequency-adaptive Radiance Field Rendering

TL;DR

3DGabSplat introduces a novel 3D Gabor-based primitive for radiance field rendering, significantly improving high-frequency detail representation, efficiency, and memory usage over existing 3D Gaussian Splatting methods.

Contribution

It proposes a 3D Gabor primitive with multiple frequency responses and an efficient CUDA rasterizer, enabling adaptive, high-quality, and memory-efficient novel view synthesis.

Findings

Outperforms 3DGS in rendering quality and efficiency

Achieves up to 1.35 dB PSNR gain over 3DGS

Reduces memory consumption and number of primitives

Abstract

Recent prominence in 3D Gaussian Splatting (3DGS) has enabled real-time rendering while maintaining high-fidelity novel view synthesis. However, 3DGS resorts to the Gaussian function that is low-pass by nature and is restricted in representing high-frequency details in 3D scenes. Moreover, it causes redundant primitives with degraded training and rendering efficiency and excessive memory overhead. To overcome these limitations, we propose 3D Gabor Splatting (3DGabSplat) that leverages a novel 3D Gabor-based primitive with multiple directional 3D frequency responses for radiance field representation supervised by multi-view images. The proposed 3D Gabor-based primitive forms a filter bank incorporating multiple 3D Gabor kernels at different frequencies to enhance flexibility and efficiency in capturing fine 3D details. Furthermore, to achieve novel view rendering, an efficient CUDA-based rasterizer is developed to project the multiple directional 3D frequency components characterized by 3D Gabor-based primitives onto the 2D image plane, and a frequency-adaptive mechanism is presented for adaptive joint optimization of primitives. 3DGabSplat is scalable to be a plug-and-play kernel for seamless integration into existing 3DGS paradigms to enhance both efficiency and quality of novel view synthesis. Extensive experiments demonstrate that 3DGabSplat outperforms 3DGS and its variants using alternative primitives, and achieves state-of-the-art rendering quality across both real-world and synthetic scenes. Remarkably, we achieve up to 1.35 dB PSNR gain over 3DGS with simultaneously reduced number of primitives and memory consumption.