Controlling the False Discovery Rate for Binary Feature Selection via Knockoff

TL;DR

This paper introduces a new method for binary feature selection in regression models that effectively controls the false discovery rate, even with estimated feature distributions, and demonstrates higher power than existing methods in simulations and real data.

Contribution

It proposes a novel FDR control method for binary features that works under mild conditions and shows improved power over existing approaches.

Findings

FDR is exactly controlled at the target level when the feature distribution is known.

FDR control remains effective when the feature distribution is estimated from data.

The method outperforms competitors in simulations and real data applications, such as HIV treatment data.

Abstract

Variable selection has been widely used in data analysis for the past decades, and it becomes increasingly important in the Big Data era as there are usually hundreds of variables available in a dataset. To enhance interpretability of a model, identifying potentially relevant features is often a step before fitting all the features into a regression model. A good variable selection method should effectively control the fraction of false discoveries and ensure large enough power of its selection set. In a lot of contemporary data applications, a great portion of features are coded as binary variables. Binary features are widespread in many fields, from online controlled experiments to genome science to physical statistics. Although there has recently been a handful of literature for provable false discovery rate (FDR) control in variable selection, most of the theoretical analyses were based on some strong dependency assumption or Gaussian assumption among features. In this paper we propose a variable selection method in regression framework for selecting binary features. Under mild conditions, we show that FDR is controlled exactly under a target level in a finite sample if the underlying distribution of the binary features is known. We show in simulations that FDR control is still attained when feature distribution is estimated from data. We also provide theoretical results on the power of our variables selection method in a linear regression model or a logistic regression model. In the restricted settings where competitors exist, we show in simulations and real data application on a HIV antiretroviral therapy dataset that our method has higher power than the competitor.