SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians

TL;DR

This paper extends 3D Gaussian Splatting to unstructured in-the-wild image collections by modeling appearance and training transient Gaussians, achieving state-of-the-art rendering quality and efficiency for complex scenes.

Contribution

It introduces appearance-conditioned modeling and a new training mechanism for transient Gaussians to improve 3D scene rendering from unstructured photo collections.

Findings

Outperforms prior methods on diverse outdoor scenes

Achieves higher rendering quality with improved efficiency

Effectively handles scene occluders in unstructured data

Abstract

Implicit neural representation methods have shown impressive advancements in learning 3D scenes from unstructured in-the-wild photo collections but are still limited by the large computational cost of volumetric rendering. More recently, 3D Gaussian Splatting emerged as a much faster alternative with superior rendering quality and training efficiency, especially for small-scale and object-centric scenarios. Nevertheless, this technique suffers from poor performance on unstructured in-the-wild data. To tackle this, we extend over 3D Gaussian Splatting to handle unstructured image collections. We achieve this by modeling appearance to seize photometric variations in the rendered images. Additionally, we introduce a new mechanism to train transient Gaussians to handle the presence of scene occluders in an unsupervised manner. Experiments on diverse photo collection scenes and multi-pass acquisition of outdoor landmarks show the effectiveness of our method over prior works achieving state-of-the-art results with improved efficiency.