Learning Out of Leaders

TL;DR

This paper introduces LOL, a simple, auto-driven thresholding algorithm for high-dimensional regression that efficiently selects relevant features and guarantees consistency without complex optimization.

Contribution

The paper proposes LOL, a novel no-optimization, thresholding-based method for high-dimensional regression with proven consistency and adaptive minimax properties.

Findings

LOL achieves strong theoretical guarantees of consistency.

The method performs well in extensive computational experiments.

LOL handles high-dimensional data with no restrictions on the number of regressors.

Abstract

This paper investigates the estimation problem in a regression-type model. To be able to deal with potential high dimensions, we provide a procedure called LOL, for Learning Out of Leaders with no optimization step. LOL is an auto-driven algorithm with two thresholding steps. A first adaptive thresholding helps to select leaders among the initial regressors in order to obtain a first reduction of dimensionality. Then a second thresholding is performed on the linear regression upon the leaders. The consistency of the procedure is investigated. Exponential bounds are obtained, leading to minimax and adaptive results for a wide class of sparse parameters, with (quasi) no restriction on the number p of possible regressors. An extensive computational experiment is conducted to emphasize the practical good performances of LOL.