Deep Generative Model with Beta Bernoulli Process for Modeling and Learning Confounding Factors

TL;DR

This paper introduces a deep generative model that combines Bayesian non-parametrics with deep learning to disentangle task-relevant features from an unknown and potentially infinite set of confounding factors, including unobservable ones.

Contribution

It proposes a novel deep conditional generative model integrating Indian Buffet Process to handle unknown, unobservable confounders in complex data.

Findings

Successfully disentangles confounding factors in MNIST and ECG datasets.

Adapts to increasing complexity of data with growing confounders.

Identifies presence or absence of unobserved confounding factors.

Abstract

While deep representation learning has become increasingly capable of separating task-relevant representations from other confounding factors in the data, two significant challenges remain. First, there is often an unknown and potentially infinite number of confounding factors coinciding in the data. Second, not all of these factors are readily observable. In this paper, we present a deep conditional generative model that learns to disentangle a task-relevant representation from an unknown number of confounding factors that may grow infinitely. This is achieved by marrying the representational power of deep generative models with Bayesian non-parametric factor models, where a supervised deterministic encoder learns task-related representation and a probabilistic encoder with an Indian Buffet Process (IBP) learns the unknown number of unobservable confounding factors. We tested the presented model in two datasets: a handwritten digit dataset (MNIST) augmented with colored digits and a clinical ECG dataset with significant inter-subject variations and augmented with signal artifacts. These diverse data sets highlighted the ability of the presented model to grow with the complexity of the data and identify the absence or presence of unobserved confounding factors.