Rega-Net:Retina Gabor Attention for Deep Convolutional Neural Networks

TL;DR

Rega-Net introduces a novel attention mechanism inspired by the human retina, enlarging the receptive field of CNNs to improve accuracy in image classification and object detection tasks.

Contribution

The paper presents a new retina-inspired attention module with variable-resolution Gabor kernels that enhances CNN receptive fields and accuracy.

Findings

Achieves 79.96% top-1 accuracy on ImageNet-1K

Improves mAP by up to 3.5% on COCO2017

Demonstrates effectiveness of retina-like kernels in CNNs

Abstract

Extensive research works demonstrate that the attention mechanism in convolutional neural networks (CNNs) effectively improves accuracy. Nevertheless, few works design attention mechanisms using large receptive fields. In this work, we propose a novel attention method named Rega-net to increase CNN accuracy by enlarging the receptive field. Inspired by the mechanism of the human retina, we design convolutional kernels to resemble the non-uniformly distributed structure of the human retina. Then, we sample variable-resolution values in the Gabor function distribution and fill these values in retina-like kernels. This distribution allows essential features to be more visible in the center position of the receptive field. We further design an attention module including these retina-like kernels. Experiments demonstrate that our Rega-Net achieves 79.96% top-1 accuracy on ImageNet-1K classification and 43.1% mAP on COCO2017 object detection. The mAP of the Rega-Net increased by up to 3.5% compared to baseline networks.