The Asymptotic Cost of Complexity

TL;DR

This paper introduces a measure of learning efficiency based on the metric entropy of state spaces, analyzing how complexity impacts learning in high-dimensional models.

Contribution

It proposes a new complexity measure for non-finite state spaces and applies it to analyze learning efficiency in high-dimensional models.

Findings

Complexity is characterized by metric entropy.

Learning efficiency is determined by state space complexity.

Applied to models with high-dimensional states.

Abstract

We propose a measure of learning efficiency for non-finite state spaces. We characterize the complexity of a learning problem by the metric entropy of its state space. We then describe how learning efficiency is determined by this measure of complexity. This is, then, applied to two models where agents learn high-dimensional states.