Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS

TL;DR

This paper introduces a new initialization pipeline for sparse-view 3D Gaussian Splatting that significantly improves rendering quality by combining frequency-aware SfM, self-initialization, and point-cloud regularization.

Contribution

The authors propose a novel initialization method that enhances sparse-view 3DGS performance, surpassing prior approaches by focusing on improved seed points and regularization techniques.

Findings

Consistent performance gains on LLFF and Mip-NeRF360 datasets.

Improved coverage of low-texture regions via frequency-aware SfM.

Enhanced point cloud quality through geometric and visibility priors.

Abstract

Sparse-view 3D Gaussian Splatting (3DGS) often overfits to the training views, leading to artifacts like blurring in novel view rendering. Prior work addresses it either by enhancing the initialization (\emph{i.e.}, the point cloud from Structure-from-Motion (SfM)) or by adding training-time constraints (regularization) to the 3DGS optimization. Yet our controlled ablations reveal that initialization is the decisive factor: it determines the attainable performance band in sparse-view 3DGS, while training-time constraints yield only modest within-band improvements at extra cost. Given initialization's primacy, we focus our design there. Although SfM performs poorly under sparse views due to its reliance on feature matching, it still provides reliable seed points. Thus, building on SfM, our effort aims to supplement the regions it fails to cover as comprehensively as possible. Specifically, we design: (i) frequency-aware SfM that improves low-texture coverage via low-frequency view augmentation and relaxed multi-view correspondences; (ii) 3DGS self-initialization that lifts photometric supervision into additional points, compensating SfM-sparse regions with learned Gaussian centers; and (iii) point-cloud regularization that enforces multi-view consistency and uniform spatial coverage through simple geometric/visibility priors, yielding a clean and reliable point cloud. Our experiments on LLFF and Mip-NeRF360 demonstrate consistent gains in sparse-view settings, establishing our approach as a stronger initialization strategy. Code is available at https://github.com/zss171999645/ItG-GS.