How to Formulate and Solve Statistical Recognition and Learning Problems

TL;DR

This paper presents a unified framework for statistical recognition and learning problems, identifying improper strategies, proposing a generalized formulation, and demonstrating the superiority of the closest to optimal strategy over traditional methods.

Contribution

It introduces a comprehensive formulation that unifies recognition and learning, including zero-sample scenarios, and proposes a new optimal strategy with demonstrated advantages.

Findings

The closest to optimal strategy outperforms maximum likelihood methods in illustrative cases.

Some widely used recognition and learning approaches are shown to be improper.

A generalized framework encompasses all sample sizes, including zero.

Abstract

We formulate problems of statistical recognition and learning in a common framework of complex hypothesis testing. Based on arguments from multi-criteria optimization, we identify strategies that are improper for solving these problems and derive a common form of the remaining strategies. We show that some widely used approaches to recognition and learning are improper in this sense. We then propose a generalized formulation of the recognition and learning problem which embraces the whole range of sizes of the learning sample, including the zero size. Learning becomes a special case of recognition without learning. We define the concept of closest to optimal strategy, being a solution to the formulated problem, and describe a technique for finding such a strategy. On several illustrative cases, the strategy is shown to be superior to the widely used learning methods based on maximal likelihood estimation.