Aggregation of supports along the Lasso path

TL;DR

This paper introduces two data-driven support aggregation procedures for linear regression that satisfy oracle inequalities without design assumptions, and applies them to Lasso supports to create an estimator with optimal prediction bounds.

Contribution

The paper presents novel support aggregation methods that work with the Lasso path, achieving oracle inequalities and optimal prediction bounds under certain conditions.

Findings

Procedures satisfy oracle inequalities without design assumptions.

Aggregated estimator mimics the best Lasso estimator.

Achieves optimal prediction bounds under restricted eigenvalue condition.

Abstract

In linear regression with fixed design, we propose two procedures that aggregate a data-driven collection of supports. The collection is a subset of the $2^p$ possible supports and both its cardinality and its elements can depend on the data. The procedures satisfy oracle inequalities with no assumption on the design matrix. Then we use these procedures to aggregate the supports that appear on the regularization path of the Lasso in order to construct an estimator that mimics the best Lasso estimator. If the restricted eigenvalue condition on the design matrix is satisfied, then this estimator achieves optimal prediction bounds. Finally, we discuss the computational cost of these procedures.