Aggregation of Affine Estimators
Dong Dai, Philippe Rigollet, Lucy Xia, Tong Zhang

TL;DR
This paper investigates aggregation methods for affine estimators in fixed design regression, establishing high-probability oracle inequalities and demonstrating the effectiveness of a generalized $Q$-aggregation scheme.
Contribution
It introduces high-probability oracle inequalities for exponential weighting and $Q$-aggregation, advancing theoretical understanding of estimator aggregation.
Findings
High-probability oracle inequalities for EW aggregation.
Sharp oracle inequalities for $Q$-aggregation.
Universal aggregation achieves all known bounds.
Abstract
We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon (2012) have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced -aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods and Inference · Advanced Statistical Methods and Models · Risk and Portfolio Optimization
