From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal

TL;DR

This paper introduces a large-scale high-quality rain dataset, a novel tensor recovery method for ground truth generation, and a transformer-based deraining baseline, significantly advancing real image deraining research.

Contribution

The paper provides a new extensive dataset, a robust tensor recovery model, and a simple yet effective transformer-based deraining method, addressing key challenges in real rain removal.

Findings

The dataset contains 3000 video sequences with 1 million high-res frame pairs.

The tensor recovery model improves ground-truth quality by better separating background and rain.

The transformer-based baseline outperforms existing deraining methods.

Abstract

Learning-based image deraining methods have made great progress. However, the lack of large-scale high-quality paired training samples is the main bottleneck to hamper the real image deraining (RID). To address this dilemma and advance RID, we construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset over the existing ones are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher-quality ground-truth. Specifically, the real rains in LHP-Rain not only contain the classical rain streak/veiling/occlusion in the sky, but also the \textbf{splashing on the ground} overlooked by deraining community. Moreover, we propose a novel robust low-rank tensor recovery model to generate the GT with better separating the static background from the dynamic rain. In addition, we design a simple transformer-based single image deraining baseline, which simultaneously utilize the self-attention and cross-layer attention within the image and rain layer with discriminative feature representation. Extensive experiments verify the superiority of the proposed dataset and deraining method over state-of-the-art.