A contextual analysis of multi-layer perceptron models in classifying hand-written digits and letters: limited resources

TL;DR

This paper evaluates a simple MLP model trained with numpy on a constrained dataset, demonstrating that data mining and dimensionality reduction can significantly improve efficiency with minimal accuracy loss.

Contribution

It introduces an end-to-end numpy-based MLP approach and shows how data mining techniques enhance performance and reduce memory usage on handwritten digit and letter classification.

Findings

MLP achieves 84.83% accuracy without data mining.

PCA reduces feature space by 90%, maintaining accuracy.

Data cleaning methods reduce training data size to 32.8%.

Abstract

Classifying hand-written digits and letters has taken a big leap with the introduction of ConvNets. However, on very constrained hardware the time necessary to train such models would be high. Our main contribution is twofold. First, we extensively test an end-to-end vanilla neural network (MLP) approach in pure numpy without any pre-processing or feature extraction done beforehand. Second, we show that basic data mining operations can significantly improve the performance of the models in terms of computational time, without sacrificing much accuracy. We illustrate our claims on a simpler variant of the Extended MNIST dataset, called Balanced EMNIST dataset. Our experiments show that, without any data mining, we get increased generalization performance when using more hidden layers and regularization techniques, the best model achieving 84.83% accuracy on a test dataset. Using dimensionality reduction done by PCA we were able to increase that figure to 85.08% with only 10% of the original feature space, reducing the memory size needed by 64%. Finally, adding methods to remove possibly harmful training samples like deviation from the mean helped us to still achieve over 84% test accuracy but with only 32.8% of the original memory size for the training set. This compares favorably to the majority of literature results obtained through similar architectures. Although this approach gets outshined by state-of-the-art models, it does scale to some (AlexNet, VGGNet) trained on 50% of the same dataset.