Convolutional Neural Network with Pruning Method for Handwritten Digit Recognition

TL;DR

This paper investigates the relationship between CNN fully connected layer size and recognition accuracy for handwritten digit recognition using MNIST, exploring pruning methods and their impact on model performance.

Contribution

It introduces a pruning approach tailored for CNNs in handwritten digit recognition and analyzes how layer size and pruning thresholds affect accuracy.

Findings

Recognition accuracy peaks at a fully connected layer size of 400.

Pruning performance on CNNs is worse than on neural networks.

Increasing the angle threshold decreases layer size and accuracy.

Abstract

CNN model is a popular method for imagery analysis, so it could be utilized to recognize handwritten digits based on MNIST datasets. For higher recognition accuracy, various CNN models with different fully connected layer sizes are exploited to figure out the relationship between the CNN fully connected layer size and the recognition accuracy. Inspired by previous pruning work, we performed pruning methods of distinctiveness on CNN models and compared the pruning performance with NN models. For better pruning performances on CNN, the effect of angle threshold on the pruning performance was explored. The evaluation results show that: for the fully connected layer size, there is a threshold, so that when the layer size increases, the recognition accuracy grows if the layer size smaller than the threshold, and falls if the layer size larger than the threshold; the performance of pruning performed on CNN is worse than on NN; as pruning angle threshold increases, the fully connected layer size and the recognition accuracy decreases. This paper also shows that for CNN models trained by the MNIST dataset, they are capable of handwritten digit recognition and achieve the highest recognition accuracy with fully connected layer size 400. In addition, for same dataset MNIST, CNN models work better than big, deep, simple NN models in a published paper.