Localization of VC Classes: Beyond Local Rademacher Complexities

TL;DR

This paper introduces a new localization method for binary classification using fixed points of local empirical entropy, providing tight bounds and addressing ERM optimality under bounded noise for VC classes.

Contribution

It proposes a novel complexity measure based on local empirical entropy fixed points and establishes its tight bounds and optimality implications.

Findings

New complexity measure via local empirical entropy fixed points

Tight upper bounds on VC class complexities

Minimax lower bounds involving the same complexity measure

Abstract

In this paper we introduce an alternative localization approach for binary classification that leads to a novel complexity measure: fixed points of the local empirical entropy. We show that this complexity measure gives a tight control over complexity in the upper bounds. Our results are accompanied by a novel minimax lower bound that involves the same quantity. In particular, we practically answer the question of optimality of ERM under bounded noise for general VC classes.