Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

TL;DR

This paper develops sharp PAC-Bayesian risk bounds for exponential weighting aggregation in regression, providing new sparsity oracle inequalities under general error and function distribution assumptions.

Contribution

It introduces novel sharp PAC-Bayesian bounds for exponential weighting aggregation and applies them to derive sparsity oracle inequalities in regression.

Findings

Sharp PAC-Bayesian risk bounds for exponential weights

Sparsity oracle inequalities derived from bounds

Applicable under general error and function distribution assumptions

Abstract

We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the distribution of errors and on the functions to aggregate. We then apply these results to derive sparsity oracle inequalities.