Competitive Multi-scale Convolution

TL;DR

This paper presents a novel convolutional module that uses competitive pooling with maxout units among multi-scale filters, improving training and performance in deep ConvNets for image classification tasks.

Contribution

Introduces a competitive multi-scale convolutional module replacing inception's pooling with maxout, enhancing training and accuracy in deep ConvNets.

Findings

Achieves state-of-the-art or comparable results on MNIST, CIFAR-10, CIFAR-100, and SVHN datasets.

Prevents filter co-adaptation and facilitates training of complex models.

Reduces output dimensionality of multi-scale filters.

Abstract

In this paper, we introduce a new deep convolutional neural network (ConvNet) module that promotes competition among a set of multi-scale convolutional filters. This new module is inspired by the inception module, where we replace the original collaborative pooling stage (consisting of a concatenation of the multi-scale filter outputs) by a competitive pooling represented by a maxout activation unit. This extension has the following two objectives: 1) the selection of the maximum response among the multi-scale filters prevents filter co-adaptation and allows the formation of multiple sub-networks within the same model, which has been shown to facilitate the training of complex learning problems; and 2) the maxout unit reduces the dimensionality of the outputs from the multi-scale filters. We show that the use of our proposed module in typical deep ConvNets produces classification results that are either better than or comparable to the state of the art on the following benchmark datasets: MNIST, CIFAR-10, CIFAR-100 and SVHN.