GhostNetV2: Enhance Cheap Operation with Long-Range Attention

TL;DR

GhostNetV2 introduces a hardware-efficient long-range attention mechanism to enhance lightweight CNNs, improving global feature capture without sacrificing speed, and achieves superior accuracy on ImageNet.

Contribution

The paper proposes DFC attention, a fast, hardware-friendly long-range attention mechanism, and integrates it into GhostNetV2 to improve global feature aggregation in mobile neural networks.

Findings

Achieves 75.3% top-1 accuracy on ImageNet with 167M FLOPs.

Outperforms GhostNetV1 with similar computational cost.

Demonstrates the effectiveness of long-range attention in lightweight CNNs.

Abstract

Light-weight convolutional neural networks (CNNs) are specially designed for applications on mobile devices with faster inference speed. The convolutional operation can only capture local information in a window region, which prevents performance from being further improved. Introducing self-attention into convolution can capture global information well, but it will largely encumber the actual speed. In this paper, we propose a hardware-friendly attention mechanism (dubbed DFC attention) and then present a new GhostNetV2 architecture for mobile applications. The proposed DFC attention is constructed based on fully-connected layers, which can not only execute fast on common hardware but also capture the dependence between long-range pixels. We further revisit the expressiveness bottleneck in previous GhostNet and propose to enhance expanded features produced by cheap operations with DFC attention, so that a GhostNetV2 block can aggregate local and long-range information simultaneously. Extensive experiments demonstrate the superiority of GhostNetV2 over existing architectures. For example, it achieves 75.3% top-1 accuracy on ImageNet with 167M FLOPs, significantly suppressing GhostNetV1 (74.5%) with a similar computational cost. The source code will be available at https://github.com/huawei-noah/Efficient-AI-Backbones/tree/master/ghostnetv2_pytorch and https://gitee.com/mindspore/models/tree/master/research/cv/ghostnetv2.