Competitive Inner-Imaging Squeeze and Excitation for Residual Network

TL;DR

This paper introduces a novel competitive squeeze-and-excitation mechanism for residual networks that enhances channel relationship modeling by jointly considering residual and identity mappings, leading to improved performance.

Contribution

It proposes an inner-imaging competitive SE block that models channel relations with convolution, expanding residual network capabilities and achieving state-of-the-art results.

Findings

Outperforms existing methods on CIFAR, SVHN, and ImageNet datasets.

Effectively models channel-wise relations with convolution in spatial.

Enhances residual network efficiency and accuracy.

Abstract

Residual networks, which use a residual unit to supplement the identity mappings, enable very deep convolutional architecture to operate well, however, the residual architecture has been proved to be diverse and redundant, which may leads to low-efficient modeling. In this work, we propose a competitive squeeze-excitation (SE) mechanism for the residual network. Re-scaling the value for each channel in this structure will be determined by the residual and identity mappings jointly, and this design enables us to expand the meaning of channel relationship modeling in residual blocks. Modeling of the competition between residual and identity mappings cause the identity flow to control the complement of the residual feature maps for itself. Furthermore, we design a novel inner-imaging competitive SE block to shrink the consumption and re-image the global features of intermediate network structure, by using the inner-imaging mechanism, we can model the channel-wise relations with convolution in spatial. We carry out experiments on the CIFAR, SVHN, and ImageNet datasets, and the proposed method can challenge state-of-the-art results.