FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training

TL;DR

FewViewGS introduces a multi-stage training approach with matching-based constraints and locality regularization for accurate novel view synthesis from sparse images, outperforming existing methods.

Contribution

The paper presents a novel 3D Gaussian-based view synthesis method that does not rely on pre-trained models and effectively handles sparse input images.

Findings

Achieves superior results on synthetic and real-world datasets.

Effectively preserves local color structure and reduces artifacts.

Outperforms state-of-the-art methods in few-shot scenarios.

Abstract

The field of novel view synthesis from images has seen rapid advancements with the introduction of Neural Radiance Fields (NeRF) and more recently with 3D Gaussian Splatting. Gaussian Splatting became widely adopted due to its efficiency and ability to render novel views accurately. While Gaussian Splatting performs well when a sufficient amount of training images are available, its unstructured explicit representation tends to overfit in scenarios with sparse input images, resulting in poor rendering performance. To address this, we present a 3D Gaussian-based novel view synthesis method using sparse input images that can accurately render the scene from the viewpoints not covered by the training images. We propose a multi-stage training scheme with matching-based consistency constraints imposed on the novel views without relying on pre-trained depth estimation or diffusion models. This is achieved by using the matches of the available training images to supervise the generation of the novel views sampled between the training frames with color, geometry, and semantic losses. In addition, we introduce a locality preserving regularization for 3D Gaussians which removes rendering artifacts by preserving the local color structure of the scene. Evaluation on synthetic and real-world datasets demonstrates competitive or superior performance of our method in few-shot novel view synthesis compared to existing state-of-the-art methods.