MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo

TL;DR

This paper introduces a novel online multi-view stereo framework for high-quality 3D Gaussian Splatting mapping, significantly improving detailed scene reconstruction and rendering accuracy in neural rendering applications.

Contribution

It presents a new method combining MVS depth estimation with 3D Gaussian Splatting, including depth refinement and efficient backend optimization, for superior real-time 3D scene modeling.

Findings

Outperforms state-of-the-art dense SLAM methods

Achieves high-quality detailed 3D reconstructions

Excels in challenging outdoor environments

Abstract

This study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most studies focus primarily on estimating coarse 3D scenes rather than achieving detailed reconstructions. Moreover, depth estimation based solely on images is often ambiguous, resulting in low-quality 3D models that lead to inaccurate renderings. To overcome these limitations, we propose a novel framework for high-quality 3DGS modeling that leverages an online multi-view stereo (MVS) approach. Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques to filter out outliers, enabling accurate initialization of Gaussians in 3DGS. Furthermore, we introduce a parallelized backend module that optimizes the 3DGS model efficiently, ensuring timely updates with each new keyframe. Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods, particularly excelling in challenging outdoor environments.