Channel-wise and Spatial Feature Modulation Network for Single Image Super-Resolution

TL;DR

This paper introduces a novel neural network architecture for single image super-resolution that employs channel-wise and spatial feature modulation to enhance informative features and preserve long-term information, outperforming existing methods.

Contribution

The proposed CSFM network with feature modulation memory modules and gated fusion introduces a new way to dynamically enhance and fuse features for improved super-resolution performance.

Findings

Outperforms state-of-the-art super-resolution methods on benchmark datasets.

Effectively enhances informative features while suppressing redundant information.

Maintains long-term information through gated fusion and dense connections.

Abstract

The performance of single image super-resolution has achieved significant improvement by utilizing deep convolutional neural networks (CNNs). The features in deep CNN contain different types of information which make different contributions to image reconstruction. However, most CNN-based models lack discriminative ability for different types of information and deal with them equally, which results in the representational capacity of the models being limited. On the other hand, as the depth of neural networks grows, the long-term information coming from preceding layers is easy to be weaken or lost in late layers, which is adverse to super-resolving image. To capture more informative features and maintain long-term information for image super-resolution, we propose a channel-wise and spatial feature modulation (CSFM) network in which a sequence of feature-modulation memory (FMM) modules is cascaded with a densely connected structure to transform low-resolution features to high informative features. In each FMM module, we construct a set of channel-wise and spatial attention residual (CSAR) blocks and stack them in a chain structure to dynamically modulate multi-level features in a global-and-local manner. This feature modulation strategy enables the high contribution information to be enhanced and the redundant information to be suppressed. Meanwhile, for long-term information persistence, a gated fusion (GF) node is attached at the end of the FMM module to adaptively fuse hierarchical features and distill more effective information via the dense skip connections and the gating mechanism. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate the superiority of our proposed method over the state-of-the-art methods.