Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors

TL;DR

This paper introduces PFGS360, a pose-free omnidirectional 3D Gaussian Splatting method that reconstructs 3D scenes from unposed videos using a novel spherical pose estimation and depth-aware densification, enabling photorealistic view synthesis.

Contribution

The paper presents a novel pose-free 3D Gaussian Splatting approach for 360-degree videos, with a spherical pose estimation module and a depth-aware densification technique, improving reconstruction accuracy and rendering quality.

Findings

Outperforms existing pose-free 3DGS methods on real-world videos.

Achieves photorealistic novel view synthesis with efficient densification.

Demonstrates robustness on synthetic datasets.

Abstract

Omnidirectional 3D Gaussian Splatting with panoramas is a key technique for 3D scene representation, and existing methods typically rely on slow SfM to provide camera poses and sparse points priors. In this work, we propose a pose-free omnidirectional 3DGS method, named PFGS360, that reconstructs 3D Gaussians from unposed omnidirectional videos. To achieve accurate camera pose estimation, we first construct a spherical consistency-aware pose estimation module, which recovers poses by establishing consistent 2D-3D correspondences between the reconstructed Gaussians and the unposed images using Gaussians' internal depth priors. Besides, to enhance the fidelity of novel view synthesis, we introduce a depth-inlier-aware densification module to extract depth inliers and Gaussian outliers with consistent monocular depth priors, enabling efficient Gaussian densification and achieving photorealistic novel view synthesis. The experiments show significant outperformance over existing pose-free and pose-aware 3DGS methods on both real-world and synthetic 360-degree videos. Code is available at https://github.com/zcq15/PFGS360.