Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness

TL;DR

This paper introduces CoMP, a novel method using a mixture of posteriors within a Conditional VAE framework to improve counterfactual inference, data integration, and fairness by enforcing independence in latent representations.

Contribution

The paper proposes CoMP, a new approach with a mixture of posteriors and a novel penalty, offering theoretical guarantees and state-of-the-art results in biological data analysis and fairness tasks.

Findings

State-of-the-art performance in tumor sample alignment and transcriptome prediction

Theoretical proof of counterfactual identifiability under certain conditions

Effective batch correction in single-cell RNA sequencing data

Abstract

Learning meaningful representations of data that can address challenges such as batch effect correction and counterfactual inference is a central problem in many domains including computational biology. Adopting a Conditional VAE framework, we show that marginal independence between the representation and a condition variable plays a key role in both of these challenges. We propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty defined in terms of mixtures of the variational posteriors to enforce this independence in latent space. We show that CoMP has attractive theoretical properties compared to previous approaches, and we prove counterfactual identifiability of CoMP under additional assumptions. We demonstrate state-of-the-art performance on a set of challenging tasks including aligning human tumour samples with cancer cell-lines, predicting transcriptome-level perturbation responses, and batch correction on single-cell RNA sequencing data. We also find parallels to fair representation learning and demonstrate that CoMP is competitive on a common task in the field.