Nonparametric Bayes inference on conditional independence

TL;DR

This paper introduces a Bayesian nonparametric method for testing conditional independence between variables, accommodating complex data types and dimensions, with applications in variable selection and criminology.

Contribution

It develops a flexible Bayesian framework using nonparametric models and conditional mutual information to test conditional independence, with proven consistency and computational algorithms.

Findings

Method effectively detects conditional dependence and independence.

Applicable to mixed discrete and continuous data.

Demonstrated through simulations and real criminology data.

Abstract

In broad applications, it is routinely of interest to assess whether there is evidence in the data to refute the assumption of conditional independence of $Y$ and $X$ conditionally on $Z$. Such tests are well developed in parametric models but are not straightforward in the nonparametric case. We propose a general Bayesian approach, which relies on an encompassing nonparametric Bayes model for the joint distribution of $Y$, $X$ and $Z$. The framework allows $Y$, $X$ and $Z$ to be random variables on arbitrary spaces, and can accommodate different dimensional vectors having a mixture of discrete and continuous measurement scales. Using conditional mutual information as a scalar summary of the strength of the conditional dependence relationship, we construct null and alternative hypotheses. We provide conditions under which the correct hypothesis will be consistently selected. Computational methods are developed, which can be incorporated within MCMC algorithms for the encompassing model. The methods are applied to variable selection and assessed through simulations and criminology applications.