Scalable Bayesian nonparametric measures for exploring pairwise dependence via Dirichlet Process Mixtures

TL;DR

This paper introduces scalable Bayesian nonparametric methods using Dirichlet Process Mixtures to detect pairwise dependence in large datasets, providing diagnostic tools for uncertainty quantification.

Contribution

It develops novel Bayesian diagnostic measures for dependence detection that are computationally feasible for large data sets, addressing limitations of traditional Bayes factor calculations.

Findings

Effective in large data sets

Provides useful diagnostic tools

Validated through simulations and real data

Abstract

In this article we propose novel Bayesian nonparametric methods using Dirichlet Process Mixture (DPM) models for detecting pairwise dependence between random variables while accounting for uncertainty in the form of the underlying distributions. A key criteria is that the procedures should scale to large data sets. In this regard we find that the formal calculation of the Bayes factor for a dependent-vs.-independent DPM joint probability measure is not feasible computationally. To address this we present Bayesian diagnostic measures for characterising evidence against a "null model" of pairwise independence. In simulation studies, as well as for a real data analysis, we show that our approach provides a useful tool for the exploratory nonparametric Bayesian analysis of large multivariate data sets.