Non-linear Convolution Filters for CNN-based Learning

TL;DR

This paper introduces a second-order convolution method inspired by visual cortex models, incorporating non-linear quadratic forms to enhance CNN expressiveness, and demonstrates improved performance on CIFAR datasets.

Contribution

It develops a novel second-order convolution technique based on Volterra kernels, extending CNNs with non-linear filters inspired by neuroscience.

Findings

Outperforms standard CNNs with linear filters on CIFAR-10 and CIFAR-100.

Achieves results competitive with state-of-the-art methods.

Shows that combining linear and non-linear filters improves classification accuracy.

Abstract

During the last years, Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in image classification. Their architectures have largely drawn inspiration by models of the primate visual system. However, while recent research results of neuroscience prove the existence of non-linear operations in the response of complex visual cells, little effort has been devoted to extend the convolution technique to non-linear forms. Typical convolutional layers are linear systems, hence their expressiveness is limited. To overcome this, various non-linearities have been used as activation functions inside CNNs, while also many pooling strategies have been applied. We address the issue of developing a convolution method in the context of a computational model of the visual cortex, exploring quadratic forms through the Volterra kernels. Such forms, constituting a more rich function space, are used as approximations of the response profile of visual cells. Our proposed second-order convolution is tested on CIFAR-10 and CIFAR-100. We show that a network which combines linear and non-linear filters in its convolutional layers, can outperform networks that use standard linear filters with the same architecture, yielding results competitive with the state-of-the-art on these datasets.