Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining

TL;DR

This paper introduces NeRD-Rain, a multi-scale Transformer-based model that effectively leverages multi-scale features and implicit neural representations for improved image deraining, demonstrating superior performance on benchmarks.

Contribution

The paper proposes a novel end-to-end multi-scale Transformer with intra-scale implicit neural representations and inter-scale bidirectional feedback for enhanced rain removal.

Findings

Outperforms state-of-the-art methods on synthetic datasets

Achieves superior results on real-world benchmarks

Robust in complex rain scenarios

Abstract

How to effectively explore multi-scale representations of rain streaks is important for image deraining. In contrast to existing Transformer-based methods that depend mostly on single-scale rain appearance, we develop an end-to-end multi-scale Transformer that leverages the potentially useful features in various scales to facilitate high-quality image reconstruction. To better explore the common degradation representations from spatially-varying rain streaks, we incorporate intra-scale implicit neural representations based on pixel coordinates with the degraded inputs in a closed-loop design, enabling the learned features to facilitate rain removal and improve the robustness of the model in complex scenarios. To ensure richer collaborative representation from different scales, we embed a simple yet effective inter-scale bidirectional feedback operation into our multi-scale Transformer by performing coarse-to-fine and fine-to-coarse information communication. Extensive experiments demonstrate that our approach, named as NeRD-Rain, performs favorably against the state-of-the-art ones on both synthetic and real-world benchmark datasets. The source code and trained models are available at https://github.com/cschenxiang/NeRD-Rain.