On aggregation for heavy-tailed classes

TL;DR

This paper proposes an aggregation method that achieves optimal error rates for heavy-tailed classes under minimal assumptions, improving learning efficiency in non-convex settings.

Contribution

It introduces a novel aggregation procedure that attains optimal error rates for heavy-tailed classes with minimal assumptions, extending learning theory.

Findings

Achieves optimal error rates for heavy-tailed classes

Works under minimal assumptions like norm equivalence and square-integrability

Provides a new approach to learning with heavy-tailed data

Abstract

We introduce an alternative to the notion of `fast rate' in Learning Theory, which coincides with the optimal error rate when the given class happens to be convex and regular in some sense. While it is well known that such a rate cannot always be attained by a learning procedure (i.e., a procedure that selects a function in the given class), we introduce an aggregation procedure that attains that rate under rather minimal assumptions -- for example, that the $L_q$ and $L_2$ norms are equivalent on the linear span of the class for some $q>2$, and the target random variable is square-integrable.