PushPull-Net: Inhibition-driven ResNet robust to image corruptions

TL;DR

This paper introduces PushPull-Conv, a new inhibition-inspired convolutional unit for ResNets, significantly improving robustness to image corruptions by mimicking visual cortex mechanisms.

Contribution

The paper proposes a novel PushPull-Conv unit inspired by visual cortex inhibition, enhancing ResNet robustness to corruptions beyond existing methods.

Findings

Achieved a new robustness benchmark with ResNet50 on ImageNet-C

PushPull-Conv combined with data augmentation improves performance

Increases stimulus selectivity and inhibition in convolutional layers

Abstract

We introduce a novel computational unit, termed PushPull-Conv, in the first layer of a ResNet architecture, inspired by the anti-phase inhibition phenomenon observed in the primary visual cortex. This unit redefines the traditional convolutional layer by implementing a pair of complementary filters: a trainable push kernel and its counterpart, the pull kernel. The push kernel (analogous to traditional convolution) learns to respond to specific stimuli, while the pull kernel reacts to the same stimuli but of opposite contrast. This configuration enhances stimulus selectivity and effectively inhibits response in regions lacking preferred stimuli. This effect is attributed to the push and pull kernels, which produce responses of comparable magnitude in such regions, thereby neutralizing each other. The incorporation of the PushPull-Conv into ResNets significantly increases their robustness to image corruption. Our experiments with benchmark corruption datasets show that the PushPull-Conv can be combined with other data augmentation techniques to further improve model robustness. We set a new robustness benchmark on ResNet50 achieving an $mCE$ of 49.95$\%$ on ImageNet-C when combining PRIME augmentation with PushPull inhibition.