A Characterization of List Regression

TL;DR

This paper extends the theoretical understanding of list learning to regression tasks by introducing new combinatorial dimensions that characterize sample complexity in realizable and agnostic settings.

Contribution

It introduces the $k$-OIG and $k$-fat-shattering dimensions, generalizing known dimensions to characterize list PAC regression.

Findings

The $k$-OIG dimension characterizes realizable list regression.

The $k$-fat-shattering dimension characterizes agnostic list regression.

Provides a complete theoretical framework for list regression sample complexity.

Abstract

There has been a recent interest in understanding and characterizing the sample complexity of list learning tasks, where the learning algorithm is allowed to make a short list of $k$ predictions, and we simply require one of the predictions to be correct. This includes recent works characterizing the PAC sample complexity of standard list classification and online list classification. Adding to this theme, in this work, we provide a complete characterization of list PAC regression. We propose two combinatorial dimensions, namely the $k$-OIG dimension and the $k$-fat-shattering dimension, and show that they characterize realizable and agnostic $k$-list regression respectively. These quantities generalize known dimensions for standard regression. Our work thus extends existing list learning characterizations from classification to regression.