Coordinate Attention for Efficient Mobile Network Design
Qibin Hou, Daquan Zhou, Jiashi Feng

TL;DR
This paper introduces coordinate attention, a novel attention mechanism that embeds positional information into channel attention for mobile networks, improving performance across classification, detection, and segmentation tasks with minimal computational cost.
Contribution
The paper proposes coordinate attention, a new attention mechanism that incorporates positional information into channel attention, enhancing mobile network performance.
Findings
Improves ImageNet classification accuracy
Enhances object detection and semantic segmentation results
Adds minimal computational overhead to mobile networks
Abstract
Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps. In this paper, we propose a novel attention mechanism for mobile networks by embedding positional information into channel attention, which we call "coordinate attention". Unlike channel attention that transforms a feature tensor to a single feature vector via 2D global pooling, the coordinate attention factorizes channel attention into two 1D feature encoding processes that aggregate features along the two spatial directions, respectively. In this way, long-range dependencies can be captured along one spatial direction and meanwhile precise positional information can be preserved…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Advanced Memory and Neural Computing · Machine Learning and ELM
MethodsCoordinate attention · Triplet Attention · Depthwise Convolution · Batch Normalization · Pointwise Convolution · Depthwise Separable Convolution · *Communicated@Fast*How Do I Communicate to Expedia? · Dropout · Squeeze-and-Excitation Block · Sigmoid Activation
