The Sample Complexity of Learning Linear Predictors with the Squared Loss

TL;DR

This paper establishes a lower bound on the number of samples needed to learn linear predictors with squared loss in an agnostic setting, highlighting fundamental limits without distributional assumptions.

Contribution

It provides a novel sample complexity lower bound for agnostic learning of linear predictors with squared loss, independent of distributional assumptions.

Findings

Lower bound on sample complexity established

Results apply to agnostic learning without distributional assumptions

Contrasts with existing results that rely on specific assumptions

Abstract

In this short note, we provide a sample complexity lower bound for learning linear predictors with respect to the squared loss. Our focus is on an agnostic setting, where no assumptions are made on the data distribution. This contrasts with standard results in the literature, which either make distributional assumptions, refer to specific parameter settings, or use other performance measures.