ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning

TL;DR

ASF-Net introduces a novel video deraining approach that leverages temporal alignment, a channel-level temporal shift module, and an adaptive learning strategy to improve performance and robustness in real-world scenarios.

Contribution

The paper proposes ASF-Net with a unique temporal shift module and a large-scale rainy video dataset, advancing adaptive learning for real-world video deraining.

Findings

Superior performance on three benchmarks

Effective in real-world scenarios

Enhanced scene adaptability

Abstract

In recent times, learning-based methods for video deraining have demonstrated commendable results. However, there are two critical challenges that these methods are yet to address: exploiting temporal correlations among adjacent frames and ensuring adaptability to unknown real-world scenarios. To overcome these challenges, we explore video deraining from a paradigm design perspective to learning strategy construction. Specifically, we propose a new computational paradigm, Alignment-Shift-Fusion Network (ASF-Net), which incorporates a temporal shift module. This module is novel to this field and provides deeper exploration of temporal information by facilitating the exchange of channel-level information within the feature space. To fully discharge the model's characterization capability, we further construct a LArge-scale RAiny video dataset (LARA) which also supports the development of this community. On the basis of the newly-constructed dataset, we explore the parameters learning process by developing an innovative re-degraded learning strategy. This strategy bridges the gap between synthetic and real-world scenes, resulting in stronger scene adaptability. Our proposed approach exhibits superior performance in three benchmarks and compelling visual quality in real-world scenarios, underscoring its efficacy. The code is available at https://github.com/vis-opt-group/ASF-Net.