Convolutional Neural Network optimization via Channel Reassessment Attention module

TL;DR

This paper introduces the Channel Reassessment Attention (CRA) module, a lightweight enhancement for CNNs that leverages spatial information in feature maps to improve performance across multiple datasets.

Contribution

The novel CRA module utilizes spatial information in channel attention, significantly boosting CNN performance with minimal computational overhead.

Findings

CRA improves accuracy on ImageNet, CIFAR, and MS COCO datasets.

CRA is lightweight and easily integrable into existing CNN architectures.

Embedding CRA consistently enhances network performance.

Abstract

The performance of convolutional neural networks (CNNs) can be improved by adjusting the interrelationship between channels with attention mechanism. However, attention mechanism in recent advance has not fully utilized spatial information of feature maps, which makes a great difference to the results of generated channel attentions. In this paper, we propose a novel network optimization module called Channel Reassessment Attention (CRA) module which uses channel attentions with spatial information of feature maps to enhance representational power of networks. We employ CRA module to assess channel attentions based on feature maps in different channels, then the final features are refined adaptively by product between channel attentions and feature maps.CRA module is a computational lightweight module and it can be embedded into any architectures of CNNs. The experiments on ImageNet, CIFAR and MS COCO datasets demonstrate that the embedding of CRA module on various networks effectively improves the performance under different evaluation standards.