Sparse Bayesian Hierarchical Modeling of High-dimensional Clustering Problems

TL;DR

This paper introduces a Bayesian hierarchical model with sparsity priors for high-dimensional clustering, effectively performing variable selection and identifying subset-specific variables, demonstrated on gene expression data.

Contribution

It presents a novel Bayesian approach using Dirichlet processes for simultaneous clustering and variable selection in high-dimensional data, with an efficient MCMC sampling scheme.

Findings

Effective variable selection in high-dimensional clustering

Improved clustering accuracy demonstrated on gene expression data

Scalable MCMC sampling scheme for complex Bayesian models

Abstract

Clustering is one of the most widely used procedures in the analysis of microarray data, for example with the goal of discovering cancer subtypes based on observed heterogeneity of genetic marks between different tissues. It is well-known that in such high-dimensional settings, the existence of many noise variables can overwhelm the few signals embedded in the high-dimensional space. We propose a novel Bayesian approach based on Dirichlet process with a sparsity prior that simultaneous performs variable selection and clustering, and also discover variables that only distinguish a subset of the cluster components. Unlike previous Bayesian formulations, we use Dirichlet process (DP) for both clustering of samples as well as for regularizing the high-dimensional mean/variance structure. To solve the computational challenge brought by this double usage of DP, we propose to make use of a sequential sampling scheme embedded within Markov chain Monte Carlo (MCMC) updates to improve the naive implementation of existing algorithms for DP mixture models. Our method is demonstrated on a simulation study and illustrated with the leukemia gene expression dataset.