Building Efficient Lightweight CNN Models

TL;DR

This paper presents a novel methodology for constructing lightweight CNNs that maintain high accuracy while significantly reducing computational and memory requirements, suitable for resource-constrained environments.

Contribution

It introduces a two-stage training approach combining dual-input-output models and transfer learning with progressive unfreezing to enhance lightweight CNN performance.

Findings

Achieved 99% accuracy on MNIST with 14,862 parameters

Reached 89% accuracy on Fashion MNIST with low model size

Attained 65% accuracy on CIFAR-10 with fewer than 20,000 parameters

Abstract

Convolutional Neural Networks (CNNs) are pivotal in image classification tasks due to their robust feature extraction capabilities. However, their high computational and memory requirements pose challenges for deployment in resource-constrained environments. This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy. The approach integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing. The dual-input-output model train on original and augmented datasets, enhancing robustness. Progressive unfreezing is applied to the unified model to optimize pre-learned features during fine-tuning, enabling faster convergence and improved model accuracy. The methodology was evaluated on three benchmark datasets; handwritten digit MNIST, fashion MNIST, and CIFAR-10. The proposed model achieved a state-of-the-art accuracy of 99% on the handwritten digit MNIST and 89% on fashion MNIST, with only 14,862 parameters and a model size of 0.17 MB. While performance on CIFAR-10 was comparatively lower (65% with less than 20,00 parameters), the results highlight the scalability of this method. The final model demonstrated fast inference times and low latency, making it suitable for real-time applications. Future directions include exploring advanced augmentation techniques, improving architectural scalability for complex datasets, and extending the methodology to tasks beyond classification. This research underscores the potential for creating efficient, scalable, and task-specific CNNs for diverse applications.