Rethinking Rainy 3D Scene Reconstruction via Perspective Transforming and Brightness Tuning

TL;DR

This paper introduces OmniRain3D, a realistic rainy 3D scene dataset, and proposes REVR-GSNet, an end-to-end framework that enhances rain removal and 3D reconstruction from degraded multi-view images.

Contribution

The paper presents a novel dataset capturing viewpoint-dependent rain effects and brightness changes, along with a unified method for rain removal and 3D scene reconstruction.

Findings

OmniRain3D improves realism in rainy 3D scene datasets.

REVR-GSNet achieves high-fidelity 3D reconstruction from rainy images.

The approach outperforms existing methods in rain removal and scene reconstruction.

Abstract

Rain degrades the visual quality of multi-view images, which are essential for 3D scene reconstruction, resulting in inaccurate and incomplete reconstruction results. Existing datasets often overlook two critical characteristics of real rainy 3D scenes: the viewpoint-dependent variation in the appearance of rain streaks caused by their projection onto 2D images, and the reduction in ambient brightness resulting from cloud coverage during rainfall. To improve data realism, we construct a new dataset named OmniRain3D that incorporates perspective heterogeneity and brightness dynamicity, enabling more faithful simulation of rain degradation in 3D scenes. Based on this dataset, we propose an end-to-end reconstruction framework named REVR-GSNet (Rain Elimination and Visibility Recovery for 3D Gaussian Splatting). Specifically, REVR-GSNet integrates recursive brightness enhancement, Gaussian primitive optimization, and GS-guided rain elimination into a unified architecture through joint alternating optimization, achieving high-fidelity reconstruction of clean 3D scenes from rain-degraded inputs. Extensive experiments show the effectiveness of our dataset and method. Our dataset and method provide a foundation for future research on multi-view image deraining and rainy 3D scene reconstruction.