Variable Selection with the Knockoffs: Composite Null Hypotheses

TL;DR

This paper extends the fixed-X knockoff framework to handle composite null hypotheses in variable selection, introducing new methods and analyzing FDR control under complex dependencies.

Contribution

It develops two novel methods, S-OLS and FRPP, for composite null inference in knockoff procedures, addressing dependent features and improving FDR control.

Findings

S-OLS and FRPP outperform BH in composite null scenarios

Analysis of FDR loss with naive knockoff application

Structural properties of test statistics under composite nulls

Abstract

The fixed-X knockoff filter is a flexible framework for variable selection with false discovery rate (FDR) control in linear models with arbitrary design matrices (of full column rank) and it allows for finite-sample selective inference via the Lasso estimates. In this paper, we extend the theory of the knockoff procedure to tests with composite null hypotheses, which are usually more relevant to real-world problems. The main technical challenge lies in handling composite nulls in tandem with dependent features from arbitrary designs. We develop two methods for composite inference with the knockoffs, namely, shifted ordinary least-squares (S-OLS) and feature-response product perturbation (FRPP), building on new structural properties of test statistics under composite nulls. We also propose two heuristic variants of S-OLS method that outperform the celebrated Benjamini-Hochberg (BH) procedure for composite nulls, which serves as a heuristic baseline under dependent test statistics. Finally, we analyze the loss in FDR when the original knockoff procedure is naively applied on composite tests.