Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models

TL;DR

This paper introduces StARS, a stability-based method for selecting regularization parameters in high-dimensional graphical models, which ensures sparse and replicable graphs without strict conditions, outperforming traditional methods.

Contribution

The paper proposes StARS, a novel stability-based approach for regularization selection in high-dimensional graph estimation, with theoretical guarantees and superior empirical performance.

Findings

StARS effectively selects regularization parameters in high-dimensional settings.

StARS achieves high probability of including all true edges in the estimated graph.

StARS outperforms K-CV, AIC, and BIC in synthetic and real data experiments.

Abstract

A challenging problem in estimating high-dimensional graphical models is to choose the regularization parameter in a data-dependent way. The standard techniques include $K$-fold cross-validation ($K$-CV), Akaike information criterion (AIC), and Bayesian information criterion (BIC). Though these methods work well for low-dimensional problems, they are not suitable in high dimensional settings. In this paper, we present StARS: a new stability-based method for choosing the regularization parameter in high dimensional inference for undirected graphs. The method has a clear interpretation: we use the least amount of regularization that simultaneously makes a graph sparse and replicable under random sampling. This interpretation requires essentially no conditions. Under mild conditions, we show that StARS is partially sparsistent in terms of graph estimation: i.e. with high probability, all the true edges will be included in the selected model even when the graph size diverges with the sample size. Empirically, the performance of StARS is compared with the state-of-the-art model selection procedures, including $K$-CV, AIC, and BIC, on both synthetic data and a real microarray dataset. StARS outperforms all these competing procedures.