On sample complexity for computational pattern recognition

TL;DR

This paper investigates the sample complexity of computable pattern recognition methods, revealing that the number of examples needed grows faster than any computable function relative to VC dimension, under no computational constraints.

Contribution

It demonstrates that for computable pattern recognition, sample complexity exceeds linear bounds and grows faster than any computable function, contrasting with classical statistical results.

Findings

Sample complexity for computable methods grows faster than any computable function.

No resource constraints are assumed on predictors or target functions.

Impossibility results in data compression inform the sample complexity bounds.

Abstract

In statistical setting of the pattern recognition problem the number of examples required to approximate an unknown labelling function is linear in the VC dimension of the target learning class. In this work we consider the question whether such bounds exist if we restrict our attention to computable pattern recognition methods, assuming that the unknown labelling function is also computable. We find that in this case the number of examples required for a computable method to approximate the labelling function not only is not linear, but grows faster (in the VC dimension of the class) than any computable function. No time or space constraints are put on the predictors or target functions; the only resource we consider is the training examples. The task of pattern recognition is considered in conjunction with another learning problem -- data compression. An impossibility result for the task of data compression allows us to estimate the sample complexity for pattern recognition.