Stepdown SLOPE for Controlled Feature Selection

TL;DR

This paper introduces stepdown-based SLOPE methods, $k$-SLOPE and F-SLOPE, to control false discovery metrics in high-dimensional feature selection, extending SLOPE's capabilities beyond FDR control.

Contribution

It proposes novel stepdown SLOPE variants for controlling $k$-FWER and FDP, with theoretical guarantees and practical guidelines for regularization parameter selection.

Findings

Theoretical guarantees for $k$-FWER and FDP control under orthogonal design.

Effective empirical performance demonstrated on simulated data.

Guidelines for choosing regularization parameters in general settings.

Abstract

Sorted L-One Penalized Estimation (SLOPE) has shown the nice theoretical property as well as empirical behavior recently on the false discovery rate (FDR) control of high-dimensional feature selection by adaptively imposing the non-increasing sequence of tuning parameters on the sorted $\ell_1$ penalties. This paper goes beyond the previous concern limited to the FDR control by considering the stepdown-based SLOPE to control the probability of $k$ or more false rejections ($k$-FWER) and the false discovery proportion (FDP). Two new SLOPEs, called $k$-SLOPE and F-SLOPE, are proposed to realize $k$-FWER and FDP control respectively, where the stepdown procedure is injected into the SLOPE scheme. For the proposed stepdown SLOPEs, we establish their theoretical guarantees on controlling $k$-FWER and FDP under the orthogonal design setting, and also provide an intuitive guideline for the choice of regularization parameter sequence in much general setting. Empirical evaluations on simulated data validate the effectiveness of our approaches on controlled feature selection and support our theoretical findings.