PCR-GS: COLMAP-Free 3D Gaussian Splatting via Pose Co-Regularizations

TL;DR

PCR-GS introduces a novel pose co-regularization approach for COLMAP-free 3D Gaussian Splatting, significantly improving scene reconstruction and camera pose estimation in complex trajectories by leveraging feature and frequency regularizations.

Contribution

It proposes a new method combining feature reprojection and wavelet-based frequency regularizations to enhance pose-free 3D scene modeling.

Findings

Outperforms existing methods in complex camera trajectory scenarios.

Achieves more accurate camera pose estimation.

Produces higher quality 3D scene reconstructions.

Abstract

COLMAP-free 3D Gaussian Splatting (3D-GS) has recently attracted increasing attention due to its remarkable performance in reconstructing high-quality 3D scenes from unposed images or videos. However, it often struggles to handle scenes with complex camera trajectories as featured by drastic rotation and translation across adjacent camera views, leading to degraded estimation of camera poses and further local minima in joint optimization of camera poses and 3D-GS. We propose PCR-GS, an innovative COLMAP-free 3DGS technique that achieves superior 3D scene modeling and camera pose estimation via camera pose co-regularization. PCR-GS achieves regularization from two perspectives. The first is feature reprojection regularization which extracts view-robust DINO features from adjacent camera views and aligns their semantic information for camera pose regularization. The second is wavelet-based frequency regularization which exploits discrepancy in high-frequency details to further optimize the rotation matrix in camera poses. Extensive experiments over multiple real-world scenes show that the proposed PCR-GS achieves superior pose-free 3D-GS scene modeling under dramatic changes of camera trajectories.