Multi-scale Convolution Aggregation and Stochastic Feature Reuse for DenseNets

TL;DR

This paper introduces a multi-scale convolution aggregation module and a stochastic feature reuse regularization to enhance DenseNets, improving accuracy and reducing overfitting with fewer parameters across multiple vision benchmarks.

Contribution

The paper proposes a novel multi-scale convolution aggregation module and a stochastic feature reuse regularization for DenseNets, leading to better performance and efficiency.

Findings

Improved accuracy on CIFAR-10, CIFAR-100, and SVHN benchmarks.

Reduced overfitting through stochastic feature reuse.

Achieved higher efficiency with fewer parameters.

Abstract

Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all subsequent layers has strong impact on performance. To feed even richer information into the network, a novel adaptive Multi-scale Convolution Aggregation module is presented in this paper. Composed of layers for multi-scale convolutions, trainable cross-scale aggregation, maxout, and concatenation, this module is highly non-linear and can boost the accuracy of DenseNet while using much fewer parameters. In addition, due to high model complexity, the network with extremely dense feature reuse is prone to overfitting. To address this problem, a regularization method named Stochastic Feature Reuse is also presented. Through randomly dropping a set of feature maps to be reused for each mini-batch during the training phase, this regularization method reduces training costs and prevents co-adaptation. Experimental results on CIFAR-10, CIFAR-100 and SVHN benchmarks demonstrated the effectiveness of the proposed methods.