Magic ELF: Image Deraining Meets Association Learning and Transformer

TL;DR

This paper introduces Magic ELF, a novel image deraining method that combines CNN and Transformer architectures with association learning and a degradation prior, achieving superior performance with lower computational costs.

Contribution

The paper proposes a unified CNN-Transformer framework with a multi-input attention module and degradation prior for efficient and effective image deraining.

Findings

Outperforms state-of-the-art MPRNet by 0.25 dB in PSNR.

Uses only 11.7% of MPRNet's computational cost.

Achieves better deraining results with fewer parameters.

Abstract

Convolutional neural network (CNN) and Transformer have achieved great success in multimedia applications. However, little effort has been made to effectively and efficiently harmonize these two architectures to satisfy image deraining. This paper aims to unify these two architectures to take advantage of their learning merits for image deraining. In particular, the local connectivity and translation equivariance of CNN and the global aggregation ability of self-attention (SA) in Transformer are fully exploited for specific local context and global structure representations. Based on the observation that rain distribution reveals the degradation location and degree, we introduce degradation prior to help background recovery and accordingly present the association refinement deraining scheme. A novel multi-input attention module (MAM) is proposed to associate rain perturbation removal and background recovery. Moreover, we equip our model with effective depth-wise separable convolutions to learn the specific feature representations and trade off computational complexity. Extensive experiments show that our proposed method (dubbed as ELF) outperforms the state-of-the-art approach (MPRNet) by 0.25 dB on average, but only accounts for 11.7\% and 42.1\% of its computational cost and parameters. The source code is available at https://github.com/kuijiang94/Magic-ELF.