Lightweight Residual Densely Connected Convolutional Neural Network

TL;DR

This paper introduces a lightweight residual densely connected CNN architecture that reduces parameters and computational costs, making it suitable for resource-constrained devices while maintaining competitive accuracy across multiple datasets.

Contribution

The paper proposes a novel lightweight residual densely connected block that enhances deep supervision, gradient flow, and feature reuse without requiring special hardware, outperforming some existing efficient models.

Findings

Outperforms AlexNet and VGGNet in size and accuracy

Achieves state-of-the-art on Fashion MNIST

Comparable to CondenseNet and ShuffleNet in efficiency

Abstract

Extremely efficient convolutional neural network architectures are one of the most important requirements for limited-resource devices (such as embedded and mobile devices). The computing power and memory size are two important constraints of these devices. Recently, some architectures have been proposed to overcome these limitations by considering specific hardware-software equipment. In this paper, the lightweight residual densely connected blocks are proposed to guaranty the deep supervision, efficient gradient flow, and feature reuse abilities of convolutional neural network. The proposed method decreases the cost of training and inference processes without using any special hardware-software equipment by just reducing the number of parameters and computational operations while achieving a feasible accuracy. Extensive experimental results demonstrate that the proposed architecture is more efficient than the AlexNet and VGGNet in terms of model size, required parameters, and even accuracy. The proposed model has been evaluated on the ImageNet, MNIST, Fashion MNIST, SVHN, CIFAR-10, and CIFAR-100. It achieves state-of-the-art results on Fashion MNIST dataset and reasonable results on the others. The obtained results show the superiority of the proposed method to efficient models such as the SqueezNet. It is also comparable with state-of-the-art efficient models such as CondenseNet and ShuffleNet.