Confidence-Based Mesh Extraction from 3D Gaussians

TL;DR

This paper introduces a confidence-based self-supervised framework for 3D Gaussian Splatting that improves mesh extraction accuracy, especially in scenes with view-dependent effects, while maintaining efficiency.

Contribution

It proposes a novel confidence-driven approach with new loss functions, enhancing surface extraction without relying on multi-view or large pre-trained models.

Findings

Achieves state-of-the-art results for unbounded mesh extraction.

Demonstrates benefits of confidence values in balancing supervision.

Improves appearance modeling by decoupling D-SSIM loss terms.

Abstract

Recently, 3D Gaussian Splatting (3DGS) greatly accelerated mesh extraction from posed images due to its explicit representation and fast software rasterization. While the addition of geometric losses and other priors has improved the accuracy of extracted surfaces, mesh extraction remains difficult in scenes with abundant view-dependent effects. To resolve the resulting ambiguities, prior works rely on multi-view techniques, iterative mesh extraction, or large pre-trained models, sacrificing the inherent efficiency of 3DGS. In this work, we present a simple and efficient alternative by introducing a self-supervised confidence framework to 3DGS: within this framework, learnable confidence values dynamically balance photometric and geometric supervision. Extending our confidence-driven formulation, we introduce losses which penalize per-primitive color and normal variance and demonstrate their benefits to surface extraction. Finally, we complement the above with an improved appearance model, by decoupling the individual terms of the D-SSIM loss. Our final approach delivers state-of-the-art results for unbounded meshes while remaining highly efficient.