Contextual Convolutional Neural Networks

TL;DR

This paper introduces Contextual Convolution (CoConv), a novel convolutional operation inspired by neuroscience, that enhances visual recognition and generative tasks by incorporating contextual information without increasing computational costs.

Contribution

We propose CoConv, a new convolutional method that integrates contextual cues into neural networks, improving performance on recognition and generative benchmarks.

Findings

Improved accuracy on ImageNet classification

Enhanced object detection on MS COCO

Better generative results on CIFAR-10 and CelebA

Abstract

We propose contextual convolution (CoConv) for visual recognition. CoConv is a direct replacement of the standard convolution, which is the core component of convolutional neural networks. CoConv is implicitly equipped with the capability of incorporating contextual information while maintaining a similar number of parameters and computational cost compared to the standard convolution. CoConv is inspired by neuroscience studies indicating that (i) neurons, even from the primary visual cortex (V1 area), are involved in detection of contextual cues and that (ii) the activity of a visual neuron can be influenced by the stimuli placed entirely outside of its theoretical receptive field. On the one hand, we integrate CoConv in the widely-used residual networks and show improved recognition performance over baselines on the core tasks and benchmarks for visual recognition, namely image classification on the ImageNet data set and object detection on the MS COCO data set. On the other hand, we introduce CoConv in the generator of a state-of-the-art Generative Adversarial Network, showing improved generative results on CIFAR-10 and CelebA. Our code is available at https://github.com/iduta/coconv.