Learning Deep Interleaved Networks with Asymmetric Co-Attention for Image Restoration

TL;DR

This paper introduces a deep interleaved network with asymmetric co-attention for image restoration, enabling multi-path feature fusion and adaptive feature emphasis to improve reconstruction quality across various IR tasks.

Contribution

It proposes a novel deep interleaved network architecture with asymmetric co-attention, enhancing feature integration and discriminative ability for image restoration.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Effectively integrates multi-level features for better image quality

Demonstrates versatility across different IR tasks

Abstract

Recently, convolutional neural network (CNN) has demonstrated significant success for image restoration (IR) tasks (e.g., image super-resolution, image deblurring, rain streak removal, and dehazing). However, existing CNN based models are commonly implemented as a single-path stream to enrich feature representations from low-quality (LQ) input space for final predictions, which fail to fully incorporate preceding low-level contexts into later high-level features within networks, thereby producing inferior results. In this paper, we present a deep interleaved network (DIN) that learns how information at different states should be combined for high-quality (HQ) images reconstruction. The proposed DIN follows a multi-path and multi-branch pattern allowing multiple interconnected branches to interleave and fuse at different states. In this way, the shallow information can guide deep representative features prediction to enhance the feature expression ability. Furthermore, we propose asymmetric co-attention (AsyCA) which is attached at each interleaved node to model the feature dependencies. Such AsyCA can not only adaptively emphasize the informative features from different states, but also improves the discriminative ability of networks. Our presented DIN can be trained end-to-end and applied to various IR tasks. Comprehensive evaluations on public benchmarks and real-world datasets demonstrate that the proposed DIN perform favorably against the state-of-the-art methods quantitatively and qualitatively.