Resource-bounded Dimension in Computational Learning Theory

TL;DR

This paper explores the connection between computational learning theory and resource-bounded dimension, establishing how learnability relates to the effective dimension of concept classes and deriving implications for complexity and unpredictability.

Contribution

It introduces new results linking resource-bounded dimension to learnability, including tight bounds for mistake-bound classes and zero polynomial-space dimension for PAC and membership-query learnable classes.

Findings

Dimension of online mistake-bound learnable classes is tightly characterized.

Polynomial-space dimension of PAC learnable classes is zero.

Polynomial-space dimension of classes learnable by membership queries is zero.

Abstract

This paper focuses on the relation between computational learning theory and resource-bounded dimension. We intend to establish close connections between the learnability/nonlearnability of a concept class and its corresponding size in terms of effective dimension, which will allow the use of powerful dimension techniques in computational learning and viceversa, the import of learning results into complexity via dimension. Firstly, we obtain a tight result on the dimension of online mistake-bound learnable classes. Secondly, in relation with PAC learning, we show that the polynomial-space dimension of PAC learnable classes of concepts is zero. This provides a hypothesis on effective dimension that implies the inherent unpredictability of concept classes (the classes that verify this property are classes not efficiently PAC learnable using any hypothesis). Thirdly, in relation to space dimension of classes that are learnable by membership query algorithms, the main result proves that polynomial-space dimension of concept classes learnable by a membership-query algorithm is zero.