Self-Distilled Disentangled Learning for Counterfactual Prediction

TL;DR

This paper introduces SD^2, a self-distilled disentanglement framework that improves counterfactual prediction accuracy by learning independent representations without complex mutual information estimators, validated on synthetic and real data.

Contribution

The paper proposes a novel information-theoretic framework, SD^2, for disentangled representation learning that simplifies mutual information minimization in high-dimensional spaces.

Findings

Effective counterfactual inference with observed and unobserved confounders

Outperforms existing methods on synthetic datasets

Validated on real-world datasets

Abstract

The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as $SD^2$. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.