On Stacked Denoising Autoencoder based Pre-training of ANN for Isolated Handwritten Bengali Numerals Dataset Recognition

TL;DR

This paper explores the use of stacked denoising autoencoders for pre-training deep neural networks to recognize handwritten Bengali numerals, achieving the lowest error rate to date on this dataset.

Contribution

It introduces the first application of SDA pre-training for Bengali numeral recognition and identifies optimal network configurations for this task.

Findings

Minimum validation error of 2.34% achieved

Optimal network has 5+ hidden layers with sigmoid activation

Hidden layer size of 1500+ neurons is recommended

Abstract

This work attempts to find the most optimal parameter setting of a deep artificial neural network (ANN) for Bengali digit dataset by pre-training it using stacked denoising autoencoder (SDA). Although SDA based recognition is hugely popular in image, speech and language processing related tasks among the researchers, it was never tried in Bengali dataset recognition. For this work, a dataset of 70000 handwritten samples were used from (Chowdhury and Rahman, 2016) and was recognized using several settings of network architecture. Among all these settings, the most optimal setting being found to be five or more deeper hidden layers with sigmoid activation and one output layer with softmax activation. We proposed the optimal number of neurons that can be used in the hidden layer is 1500 or more. The minimum validation error found from this work is 2.34% which is the lowest error rate on handwritten Bengali dataset proposed till date.