Learning Image Deraining Transformer Network with Dynamic Dual Self-Attention

TL;DR

This paper introduces a novel Transformer-based image deraining network that combines dynamic dual self-attention with a spatial-enhanced feed-forward network to improve the quality of derained images by better modeling relevant features.

Contribution

It proposes a new dynamic dual self-attention mechanism that combines dense and sparse attention, along with a spatial-enhanced feed-forward network for superior image deraining.

Findings

Outperforms existing methods on benchmark datasets.

Effectively models relevant features with dual attention.

Produces high-quality derained images with reduced blurring.

Abstract

Recently, Transformer-based architecture has been introduced into single image deraining task due to its advantage in modeling non-local information. However, existing approaches tend to integrate global features based on a dense self-attention strategy since it tend to uses all similarities of the tokens between the queries and keys. In fact, this strategy leads to ignoring the most relevant information and inducing blurry effect by the irrelevant representations during the feature aggregation. To this end, this paper proposes an effective image deraining Transformer with dynamic dual self-attention (DDSA), which combines both dense and sparse attention strategies to better facilitate clear image reconstruction. Specifically, we only select the most useful similarity values based on top-k approximate calculation to achieve sparse attention. In addition, we also develop a novel spatial-enhanced feed-forward network (SEFN) to further obtain a more accurate representation for achieving high-quality derained results. Extensive experiments on benchmark datasets demonstrate the effectiveness of our proposed method.