Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVS

TL;DR

This paper introduces a correspondence-guided SfM-free 3D Gaussian splatting method for novel view synthesis that improves pose alignment and optimization stability without relying on traditional structure-from-motion techniques.

Contribution

It proposes a novel correspondence-guided approach that enhances pose estimation and scene optimization in SfM-free NVS using 3D Gaussian splatting.

Findings

Outperforms state-of-the-art methods in accuracy.

Achieves faster convergence and better stability.

Demonstrates superior visual quality in synthesized views.

Abstract

Novel View Synthesis (NVS) without Structure-from-Motion (SfM) pre-processed camera poses--referred to as SfM-free methods--is crucial for promoting rapid response capabilities and enhancing robustness against variable operating conditions. Recent SfM-free methods have integrated pose optimization, designing end-to-end frameworks for joint camera pose estimation and NVS. However, most existing works rely on per-pixel image loss functions, such as L2 loss. In SfM-free methods, inaccurate initial poses lead to misalignment issue, which, under the constraints of per-pixel image loss functions, results in excessive gradients, causing unstable optimization and poor convergence for NVS. In this study, we propose a correspondence-guided SfM-free 3D Gaussian splatting for NVS. We use correspondences between the target and the rendered result to achieve better pixel alignment, facilitating the optimization of relative poses between frames. We then apply the learned poses to optimize the entire scene. Each 2D screen-space pixel is associated with its corresponding 3D Gaussians through approximated surface rendering to facilitate gradient back propagation. Experimental results underline the superior performance and time efficiency of the proposed approach compared to the state-of-the-art baselines.