Massively Deep Artificial Neural Networks for Handwritten Digit Recognition

TL;DR

This paper demonstrates that deep neural networks with many layers and neurons, trained efficiently using graphics cards, can achieve state-of-the-art accuracy in handwritten digit recognition on the MNIST dataset.

Contribution

It shows that increasing network depth and size, combined with GPU acceleration, significantly improves recognition accuracy on MNIST.

Findings

Achieved 0.72% error rate on MNIST.

Deep networks with many hidden layers are effective.

GPU acceleration speeds up training process.

Abstract

Greedy Restrictive Boltzmann Machines yield an fairly low 0.72% error rate on the famous MNIST database of handwritten digits. All that was required to achieve this result was a high number of hidden layers consisting of many neurons, and a graphics card to greatly speed up the rate of learning.