Local Risk Bounds for Statistical Aggregation

TL;DR

This paper improves theoretical bounds in statistical aggregation by introducing local complexity measures, leading to tighter risk bounds for key estimators in regression problems.

Contribution

It replaces global complexity with local measures in aggregation bounds, enhancing the precision of risk estimates in statistical learning.

Findings

Localized bounds for exponential weights estimator

Deviation-optimal bounds for Q-aggregation estimator

Improved results over previous bounds in regression settings

Abstract

In the problem of aggregation, the aim is to combine a given class of base predictors to achieve predictions nearly as accurate as the best one. In this flexible framework, no assumption is made on the structure of the class or the nature of the target. Aggregation has been studied in both sequential and statistical contexts. Despite some important differences between the two problems, the classical results in both cases feature the same global complexity measure. In this paper, we revisit and tighten classical results in the theory of aggregation in the statistical setting by replacing the global complexity with a smaller, local one. Some of our proofs build on the PAC-Bayes localization technique introduced by Catoni. Among other results, we prove localized versions of the classical bound for the exponential weights estimator due to Leung and Barron and deviation-optimal bounds for the Q-aggregation estimator. These bounds improve over the results of Dai, Rigollet and Zhang for fixed design regression and the results of Lecu\'e and Rigollet for random design regression.