Lasso, knockoff and Gaussian covariates: a comparison

TL;DR

This paper compares lasso, knockoff, and Gaussian covariate methods for variable selection in linear regression, demonstrating that the Gaussian covariate approach is faster, simpler, and more accurate across simulations and real data.

Contribution

The paper introduces and validates a model-free Gaussian covariate method that outperforms existing techniques in variable selection tasks.

Findings

Gaussian covariate method is faster and simpler.

It provides more accurate variable selection.

It outperforms lasso and knockoff in all tested scenarios.

Abstract

Given data $\mathbf{y}$ and $k$ covariates $\mathbf{x}_j$ one problem in linear regression is to decide which if any of the covariates to include when regressing the dependent variable $\mathbf{y}$ on the covariates $\mathbf{x}_j$. In this paper three such methods, lasso, knockoff and Gaussian covariates are compared using simulations and real data. The Gaussian covariate method is based on exact probabilities which are valid for all $\mathbf{y}$ and $\mathbf{x}_j$ making it model free. Moreover the probabilities agree with those based on the F-distribution for the standard linear model with i.i.d. Gaussian errors. It is conceptually, mathematically and algorithmically very simple, it is very fast and makes no use of simulations. It outperforms lasso and knockoff in all respects by a considerable margin.