Comparison and Combination of State-of-the-art Techniques for Handwritten Character Recognition: Topping the MNIST Benchmark

TL;DR

This paper compares four state-of-the-art handwritten digit recognition systems on MNIST, demonstrating that combining their outputs significantly reduces error rates to a new best of 0.35%, highlighting the benefits of classifier fusion.

Contribution

It introduces a method of combining multiple top-performing recognition systems to achieve the lowest error rate on MNIST, surpassing previous results.

Findings

Combined system achieves 0.35% error rate on MNIST

Errors differ among individual classifiers, enabling effective fusion

Statistical analysis confirms significance of combined results

Abstract

Although the recognition of isolated handwritten digits has been a research topic for many years, it continues to be of interest for the research community and for commercial applications. We show that despite the maturity of the field, different approaches still deliver results that vary enough to allow improvements by using their combination. We do so by choosing four well-motivated state-of-the-art recognition systems for which results on the standard MNIST benchmark are available. When comparing the errors made, we observe that the errors made differ between all four systems, suggesting the use of classifier combination. We then determine the error rate of a hypothetical system that combines the output of the four systems. The result obtained in this manner is an error rate of 0.35% on the MNIST data, the best result published so far. We furthermore discuss the statistical significance of the combined result and of the results of the individual classifiers.