Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs

TL;DR

This paper introduces a novel method for improving 3D Gaussian Splatting by dynamically integrating stereo-derived depth cues during training, leading to more accurate scene geometry and reduced artifacts.

Contribution

It presents a new strategy for incorporating depth priors from stereo networks into Gaussian Splatting, enabling self-improvement of 3D scene representations during training.

Findings

Enhanced depth accuracy demonstrated on three datasets.

First assessment of depth accuracy for Gaussian Splatting models.

Significant reduction in rendering artifacts.

Abstract

3D Gaussian Splatting (GS) significantly struggles to accurately represent the underlying 3D scene geometry, resulting in inaccuracies and floating artifacts when rendering depth maps. In this paper, we address this limitation, undertaking a comprehensive analysis of the integration of depth priors throughout the optimization process of Gaussian primitives, and present a novel strategy for this purpose. This latter dynamically exploits depth cues from a readily available stereo network, processing virtual stereo pairs rendered by the GS model itself during training and achieving consistent self-improvement of the scene representation. Experimental results on three popular datasets, breaking ground as the first to assess depth accuracy for these models, validate our findings.