Caveats on Bayesian and hidden-Markov models (v2.8)

TL;DR

This paper discusses fundamental issues in applying Bayesian and hidden-Markov models to cursive-script recognition, highlighting error propagation, the Markov assumption's validity, and alternative methods with promising results.

Contribution

It identifies key problems in Bayesian and HMM applications, provides Monte Carlo error analysis, and explores non-Bayesian, non-Markov handwriting recognition methods.

Findings

Error propagation follows a Poisson distribution over log probabilities.

Basic tests can determine the necessity of modeling serial dependencies.

Non-Bayesian, nearest-mean classification yields acceptable handwriting recognition results.

Abstract

This paper describes a number of fundamental and practical problems in the application of hidden-Markov models and Bayes when applied to cursive-script recognition. Several problems, however, will have an effect in other application areas. The most fundamental problem is the propagation of error in the product of probabilities. This is a common and pervasive problem which deserves more attention. On the basis of Monte Carlo modeling, tables for the expected relative error are given. It seems that it is distributed according to a continuous Poisson distribution over log probabilities. A second essential problem is related to the appropriateness of the Markov assumption. Basic tests will reveal whether a problem requires modeling of the stochastics of seriality, at all. Examples are given of lexical encodings which cover 95-99% classification accuracy of a lexicon, with removed sequence information, for several European languages. Finally, a summary of results on a non- Bayes, non-Markov method in handwriting recognition are presented, with very acceptable results and minimal modeling or training requirements using nearest-mean classification.