Bayesian Controlled FDR Variable Selection via Parameter-Expanded Latent Knockoffs

TL;DR

This paper introduces a Bayesian extension of the knockoff filter for variable selection that leverages Gaussian graphical models to improve FDR control and stability in identifying significant predictors.

Contribution

It develops a fully Bayesian model incorporating a latent knockoff layer and Gaussian graphical models, enhancing variable selection and FDR control over classical methods.

Findings

The method controls Bayesian FDR at any level with known covariate distribution.

It improves stability and performance compared to classical knockoff and Bayesian methods.

The approach extends to non-Gaussian responses and demonstrates effectiveness on real data.

Abstract

In many research fields, researchers aim to identify significant associations between a set of explanatory variables and a response while controlling the FDR. The Knockoff filter has been recently proposed in the frequentist paradigm to introduce controlled noise in a model by cleverly constructing copies of the predictors as auxiliary variables. We develop a fully Bayesian generalization of the classical model-X knockoff filter for normally distributed covariates. In our approach, we consider a joint model for the covariates and the response, where the conditional independence structure of the covariates is captured through a Gaussian graphical model and used to define a latent knockoff layer through a parameter-expanded representation of the response model. Estimating the covariate graph informs the knockoff construction and improves inference on the covariate effects. We use a modified spike-and-slab prior on the regression coefficients, avoiding the increase of the model dimension typical of the classical knockoff filter. We also address extensions to non-Gaussian responses. Our model performs variable selection using an upper bound on the posterior probability of non-inclusion. We show that the induced latent knockoff layer defines valid Gaussian model-X knockoffs under the proposed construction and that the resulting procedure controls the Bayesian FDR at an arbitrary level, in finite samples, if the distribution of the covariates is fully known; under an estimated graphical structure, it satisfies an asymptotic FDR guarantee. We use simulated data to demonstrate that our proposal increases the stability of the selection with respect to classical knockoff methods. With respect to Bayesian variable selection methods, our selection procedure achieves comparable or better performances, while maintaining control over the FDR. We conclude with an application to real data.