Greedy algorithms for prediction

TL;DR

This paper investigates the use of greedy algorithms for prediction in high-dimensional settings, demonstrating their consistency and optimal convergence rates, and showing they can efficiently approximate solutions like Lasso.

Contribution

It introduces and analyzes greedy algorithms for prediction with high-dimensional data, providing theoretical guarantees and improved convergence rates over existing methods.

Findings

Greedy algorithms are consistent for high-dimensional prediction.

Derived convergence rates are minimax or better, even with dependent and unbounded regressors.

Some algorithms offer fast solutions comparable to Lasso.

Abstract

In many prediction problems, it is not uncommon that the number of variables used to construct a forecast is of the same order of magnitude as the sample size, if not larger. We then face the problem of constructing a prediction in the presence of potentially large estimation error. Control of the estimation error is either achieved by selecting variables or combining all the variables in some special way. This paper considers greedy algorithms to solve this problem. It is shown that the resulting estimators are consistent under weak conditions. In particular, the derived rates of convergence are either minimax or improve on the ones given in the literature allowing for dependence and unbounded regressors. Some versions of the algorithms provide fast solution to problems such as Lasso.