WavePlanes: Compact Hex Planes for Dynamic Novel View Synthesis

TL;DR

WavePlanes introduces a compact wavelet-based hex plane representation for dynamic novel view synthesis, significantly reducing resource demands while maintaining high reconstruction quality, and enabling effective compression schemes.

Contribution

The paper proposes WavePlanes, a novel wavelet-based hex plane method that improves compactness and efficiency in dynamic NVS compared to existing approaches.

Findings

WavePlanes is significantly smaller than state-of-the-art models.

WavePlanes achieves comparable or better view synthesis quality.

WavePlanes enables effective wavelet coefficient compression.

Abstract

Dynamic Novel View Synthesis (Dynamic NVS) enhances NVS technologies to model moving 3-D scenes. However, current methods are resource intensive and challenging to compress. To address this, we present WavePlanes, a fast and more compact hex plane representation, applicable to both Neural Radiance Fields and Gaussian Splatting methods. Rather than modeling many feature scales separately (as done previously), we use the inverse discrete wavelet transform to reconstruct features at varying scales. This leads to a more compact representation and allows us to explore wavelet-based compression schemes for further gains. The proposed compression scheme exploits the sparsity of wavelet coefficients, by applying hard thresholding to the wavelet planes and storing nonzero coefficients and their locations on each plane in a Hash Map. Compared to the state-of-the-art (SotA), WavePlanes is significantly smaller, less resource demanding and competitive in reconstruction quality. Compared to small SotA models, WavePlanes outperforms methods in both model size and quality of novel views.