WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting

TL;DR

WeatherGS introduces a novel 3D scene reconstruction framework that effectively removes weather-induced artifacts from multi-view images, enabling clearer 3D reconstructions in adverse weather conditions.

Contribution

The paper proposes WeatherGS, a new method that categorizes and removes weather artifacts before 3D Gaussian Splatting, improving scene clarity under challenging weather scenarios.

Findings

Outperforms existing methods on a new weather reconstruction benchmark.

Effectively removes dense particles and lens occlusions from images.

Produces high-quality, clear 3D scenes across various adverse weather conditions.

Abstract

3D Gaussian Splatting (3DGS) has gained significant attention for 3D scene reconstruction, but still suffers from complex outdoor environments, especially under adverse weather. This is because 3DGS treats the artifacts caused by adverse weather as part of the scene and will directly reconstruct them, largely reducing the clarity of the reconstructed scene. To address this challenge, we propose WeatherGS, a 3DGS-based framework for reconstructing clear scenes from multi-view images under different weather conditions. Specifically, we explicitly categorize the multi-weather artifacts into the dense particles and lens occlusions that have very different characters, in which the former are caused by snowflakes and raindrops in the air, and the latter are raised by the precipitation on the camera lens. In light of this, we propose a dense-to-sparse preprocess strategy, which sequentially removes the dense particles by an Atmospheric Effect Filter (AEF) and then extracts the relatively sparse occlusion masks with a Lens Effect Detector (LED). Finally, we train a set of 3D Gaussians by the processed images and generated masks for excluding occluded areas, and accurately recover the underlying clear scene by Gaussian splatting. We conduct a diverse and challenging benchmark to facilitate the evaluation of 3D reconstruction under complex weather scenarios. Extensive experiments on this benchmark demonstrate that our WeatherGS consistently produces high-quality, clean scenes across various weather scenarios, outperforming existing state-of-the-art methods. See project page:https://jumponthemoon.github.io/weather-gs.