# On the support recovery of marginal regression

## TL;DR

This paper analyzes the support recovery capabilities of marginal regression in high-dimensional settings, providing a detailed characterization of its performance and factors affecting its success, especially under correlated design conditions.

## Contribution

It offers a comprehensive analysis of marginal regression's support recovery, introducing the concept of MR incoherence and comparing it with other feature selection methods.

## Key findings

- Identifies MR incoherence as key to support recovery performance.
- Provides a nuanced characterization of MR's effectiveness.
- Relates MR behavior to methods like Lasso, OMP, and SIS.

## Abstract

Leading methods for support recovery in high-dimensional regression, such as Lasso, have been well-studied and their limitations in the context of correlated design have been characterized with precise incoherence conditions. In this work, we present a similar treatment of selection consistency for marginal regression (MR), a computationally efficient family of methods with connections to decision trees. Selection based on marginal regression is also referred to as covariate screening or independence screening and is a popular approach in applied work, especially in ultra high-dimensional settings. We identify the underlying factors---which we denote as \emph{MR incoherence}---affecting MR's support recovery performance. Our near complete characterization provides a much more nuanced and optimistic view of MR in comparison to previous works. To ground our results, we provide a broad taxonomy of results for leading feature selection methods, relating the behavior of Lasso, OMP, SIS, and MR. We also lay the foundation for interesting generalizations of our analysis, e.g., to non-linear feature selection methods and to more general regression frameworks such as a general additive models.