Robust Bayesian Model Selection for Variable Clustering with the Gaussian Graphical Model

TL;DR

This paper introduces a Bayesian approach for variable clustering with Gaussian graphical models that effectively handles small noise-induced partial correlations, improving accuracy over traditional BIC-based methods especially in noisy data scenarios.

Contribution

It proposes a novel Bayesian model and evaluation method using marginal likelihood, with solutions via variational approximation and MCMC, enhancing clustering robustness in noisy environments.

Findings

More accurate than BIC in noisy settings

Provides intuitively sensible clustering on real data

Achieves similar accuracy to BIC in noise-free conditions

Abstract

Variable clustering is important for explanatory analysis. However, only few dedicated methods for variable clustering with the Gaussian graphical model have been proposed. Even more severe, small insignificant partial correlations due to noise can dramatically change the clustering result when evaluating for example with the Bayesian Information Criteria (BIC). In this work, we try to address this issue by proposing a Bayesian model that accounts for negligible small, but not necessarily zero, partial correlations. Based on our model, we propose to evaluate a variable clustering result using the marginal likelihood. To address the intractable calculation of the marginal likelihood, we propose two solutions: one based on a variational approximation, and another based on MCMC. Experiments on simulated data shows that the proposed method is similarly accurate as BIC in the no noise setting, but considerably more accurate when there are noisy partial correlations. Furthermore, on real data the proposed method provides clustering results that are intuitively sensible, which is not always the case when using BIC or its extensions.