A Bayesian model for recognizing handwritten mathematical expressions

TL;DR

This paper introduces a Bayesian probabilistic framework for recognizing handwritten mathematical expressions, capturing multiple interpretations and allowing user interaction to select the most accurate parse, outperforming previous methods.

Contribution

The paper presents a novel Bayesian model that generates a parse forest of possible interpretations, improving recognition accuracy over prior non-probabilistic systems.

Findings

Higher recognition accuracy than previous systems

Effective handling of ambiguous handwritten input

User can select preferred interpretation from multiple options

Abstract

Recognizing handwritten mathematics is a challenging classification problem, requiring simultaneous identification of all the symbols comprising an input as well as the complex two-dimensional relationships between symbols and subexpressions. Because of the ambiguity present in handwritten input, it is often unrealistic to hope for consistently perfect recognition accuracy. We present a system which captures all recognizable interpretations of the input and organizes them in a parse forest from which individual parse trees may be extracted and reported. If the top-ranked interpretation is incorrect, the user may request alternates and select the recognition result they desire. The tree extraction step uses a novel probabilistic tree scoring strategy in which a Bayesian network is constructed based on the structure of the input, and each joint variable assignment corresponds to a different parse tree. Parse trees are then reported in order of decreasing probability. Two accuracy evaluations demonstrate that the resulting recognition system is more accurate than previous versions (which used non-probabilistic methods) and other academic math recognizers.