KeyGS: A Keyframe-Centric Gaussian Splatting Method for Monocular Image Sequences

TL;DR

This paper introduces KeyGS, a fast and accurate monocular 3D reconstruction method that refines camera poses using Gaussian Splatting, reducing training time from hours to minutes.

Contribution

It proposes a novel framework combining SfM and 3D Gaussian Splatting with a coarse-to-fine densification for efficient monocular 3D reconstruction without depth or matching models.

Findings

Reduces training time from hours to minutes.

Achieves more accurate novel view synthesis.

Improves camera pose estimation accuracy.

Abstract

Reconstructing high-quality 3D models from sparse 2D images has garnered significant attention in computer vision. Recently, 3D Gaussian Splatting (3DGS) has gained prominence due to its explicit representation with efficient training speed and real-time rendering capabilities. However, existing methods still heavily depend on accurate camera poses for reconstruction. Although some recent approaches attempt to train 3DGS models without the Structure-from-Motion (SfM) preprocessing from monocular video datasets, these methods suffer from prolonged training times, making them impractical for many applications. In this paper, we present an efficient framework that operates without any depth or matching model. Our approach initially uses SfM to quickly obtain rough camera poses within seconds, and then refines these poses by leveraging the dense representation in 3DGS. This framework effectively addresses the issue of long training times. Additionally, we integrate the densification process with joint refinement and propose a coarse-to-fine frequency-aware densification to reconstruct different levels of details. This approach prevents camera pose estimation from being trapped in local minima or drifting due to high-frequency signals. Our method significantly reduces training time from hours to minutes while achieving more accurate novel view synthesis and camera pose estimation compared to previous methods.