High-dimensional Feature Selection Using Hierarchical Bayesian Logistic Regression with Heavy-tailed Priors

TL;DR

This paper presents a Bayesian logistic regression approach with heavy-tailed priors for high-dimensional feature selection, demonstrating improved accuracy in identifying relevant genes in cancer research.

Contribution

It introduces a novel Bayesian logistic regression method with heavy-tailed priors and Hamiltonian Monte Carlo for effective feature selection in high-dimensional data.

Findings

Identified 3 key genes out of 6033 candidates.

Achieved superior cross-validated prediction accuracy.

Discussed advantages and limitations of the method.

Abstract

The problem of selecting the most useful features from a great many (eg, thousands) of candidates arises in many areas of modern sciences. An interesting problem from genomic research is that, from thousands of genes that are active (expressed) in certain tissue cells, we want to find the genes that can be used to separate tissues of different classes (eg. cancer and normal). In this paper, we report our empirical experiences of using Bayesian logistic regression based on heavy-tailed priors with moderately small degree freedom (such as 1) and very small scale, and using Hamiltonian Monte Carlo to do computation. We discuss the advantages and limitations of this method, and illustrate the difficulties that remain unsolved. The method is applied to a real microarray data set related to prostate cancer. The method identifies only 3 non-redundant genes out of 6033 candidates but achieves better leave-one-out cross-validated prediction accuracy than many other methods.