TrackGS: Optimizing COLMAP-Free 3D Gaussian Splatting with Global Track Constraints

TL;DR

TrackGS integrates global feature tracks with 3D Gaussian Splatting to improve camera pose estimation and rendering quality without COLMAP preprocessing, enabling more practical and accurate novel view synthesis.

Contribution

The paper introduces a novel method that uses feature tracks to establish global constraints, optimizing camera parameters and 3D Gaussians simultaneously in COLMAP-free 3DGS.

Findings

Achieves state-of-the-art pose accuracy on real-world and synthetic datasets.

Maintains high rendering quality comparable to COLMAP-based methods.

Eliminates the need for COLMAP preprocessing, simplifying the pipeline.

Abstract

We present TrackGS, a novel method to integrate global feature tracks with 3D Gaussian Splatting (3DGS) for COLMAP-free novel view synthesis. While 3DGS delivers impressive rendering quality, its reliance on accurate precomputed camera parameters remains a significant limitation. Existing COLMAP-free approaches depend on local constraints that fail in complex scenarios. Our key innovation lies in leveraging feature tracks to establish global geometric constraints, enabling simultaneous optimization of camera parameters and 3D Gaussians. Specifically, we: (1) introduce track-constrained Gaussians that serve as geometric anchors, (2) propose novel 2D and 3D track losses to enforce multi-view consistency, and (3) derive differentiable formulations for camera intrinsics optimization. Extensive experiments on challenging real-world and synthetic datasets demonstrate state-of-the-art performance, with much lower pose error than previous methods while maintaining superior rendering quality. Our approach eliminates the need for COLMAP preprocessing, making 3DGS more accessible for practical applications.