Extensions of stability selection using subsamples of observations and covariates

TL;DR

This paper extends stability selection by applying it to random subsamples of observations and covariates, analyzing theoretical effects, and validating improvements through experiments on synthetic and real data.

Contribution

It introduces and analyzes new extensions of stability selection that incorporate random covariate subsets and arbitrary observation subsample sizes, broadening its theoretical foundation and practical applicability.

Findings

Extensions improve variable selection stability.

Theoretical analysis confirms benefits of covariate subsampling.

Numerical experiments demonstrate enhanced performance over original method.

Abstract

We introduce extensions of stability selection, a method to stabilise variable selection methods introduced by Meinshausen and B\"uhlmann (J R Stat Soc 72:417-473, 2010). We propose to apply a base selection method repeatedly to random observation subsamples and covariate subsets under scrutiny, and to select covariates based on their selection frequency. We analyse the effects and benefits of these extensions. Our analysis generalizes the theoretical results of Meinshausen and B\"uhlmann (J R Stat Soc 72:417-473, 2010) from the case of half-samples to subsamples of arbitrary size. We study, in a theoretical manner, the effect of taking random covariate subsets using a simplified score model. Finally we validate these extensions on numerical experiments on both synthetic and real datasets, and compare the obtained results in detail to the original stability selection method.