RecNets: Channel-wise Recurrent Convolutional Neural Networks

TL;DR

RecNets introduce channel-wise recurrent convolutional layers that process input channels in a recurrent manner, creating compact yet effective neural networks for image classification with improved size-accuracy trade-offs.

Contribution

The paper proposes a novel channel-wise recurrent convolutional layer and RecNets architecture, which significantly reduce parameters while maintaining high accuracy in vision tasks.

Findings

RecNets outperform other compact models on CIFAR-10 and CIFAR-100.

Channel-wise recurrent layers enable parameter sharing, reducing model size.

RecNets achieve superior size-accuracy trade-offs.

Abstract

In this paper, we introduce Channel-wise recurrent convolutional neural networks (RecNets), a family of novel, compact neural network architectures for computer vision tasks inspired by recurrent neural networks (RNNs). RecNets build upon Channel-wise recurrent convolutional (CRC) layers, a novel type of convolutional layer that splits the input channels into disjoint segments and processes them in a recurrent fashion. In this way, we simulate wide, yet compact models, since the number of parameters is vastly reduced via the parameter sharing of the RNN formulation. Experimental results on the CIFAR-10 and CIFAR-100 image classification tasks demonstrate the superior size-accuracy trade-off of RecNets compared to other compact state-of-the-art architectures.