Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

TL;DR

This paper presents a hybrid Gaussian-hash-grid radiance model that improves scene reconstruction by separating geometry and appearance, reducing artifacts, and using fewer primitives for high-fidelity novel-view synthesis.

Contribution

It introduces a novel hybrid representation with frequency-aware decomposition and pruning, enhancing scene reconstruction and rendering efficiency over prior Gaussian-based methods.

Findings

Achieves superior reconstruction fidelity compared to state-of-the-art methods.

Uses an order of magnitude fewer primitives for scene representation.

Demonstrates effectiveness on both synthetic and real-world datasets.

Abstract

We introduce a hybrid Gaussian-hash-grid radiance representation for reconstructing 2D Gaussian scene models from multi-view images. Similar to NeST splatting, our approach reduces the entanglement between geometry and appearance common in NeRF-based models, but adds per-Gaussian latent features alongside hash-grid features to bias the optimizer toward a separation of low- and high-frequency scene components. This explicit frequency-based decomposition reduces the tendency of high-frequency texture to compensate for geometric errors. Encouraging Gaussians with hard opacity falloffs further strengthens the separation between geometry and appearance, improving both geometry reconstruction and rendering efficiency. Finally, probabilistic pruning combined with a sparsity-inducing BCE opacity loss allows redundant Gaussians to be turned off, yielding a minimal set of Gaussians sufficient to represent the scene. Using both synthetic and real-world datasets, we compare against the state of the art in Gaussian-based novel-view synthesis and demonstrate superior reconstruction fidelity with an order of magnitude fewer primitives.