The Compressed Model of Residual CNDS

TL;DR

This paper introduces Residual-Squeeze-CNDS, a compressed convolutional neural network model that improves training speed and reduces size while maintaining high accuracy, building on previous residual CNDS models and leveraging a generalized compression technique.

Contribution

The paper presents a novel, more generalizable compression method for residual CNNs, integrated with residual learning to enhance speed, size, and accuracy.

Findings

Achieved 87.64% validation Top-1 accuracy on MIT Places365-Standard.

Model is 13.33% faster in training time.

Model size is significantly reduced compared to previous models.

Abstract

Convolutional neural networks have achieved a great success in the recent years. Although, the way to maximize the performance of the convolutional neural networks still in the beginning. Furthermore, the optimization of the size and the time that need to train the convolutional neural networks is very far away from reaching the researcher's ambition. In this paper, we proposed a new convolutional neural network that combined several techniques to boost the optimization of the convolutional neural network in the aspects of speed and size. As we used our previous model Residual-CNDS (ResCNDS), which solved the problems of slower convergence, overfitting, and degradation, and compressed it. The outcome model called Residual-Squeeze-CNDS (ResSquCNDS), which we demonstrated on our sold technique to add residual learning and our model of compressing the convolutional neural networks. Our model of compressing adapted from the SQUEEZENET model, but our model is more generalizable, which can be applied almost to any neural network model, and fully integrated into the residual learning, which addresses the problem of the degradation very successfully. Our proposed model trained on very large-scale MIT Places365-Standard scene datasets, which backing our hypothesis that the new compressed model inherited the best of the previous ResCNDS8 model, and almost get the same accuracy in the validation Top-1 and Top-5 with 87.64% smaller in size and 13.33% faster in the training time.