Efficiently Disentangle Causal Representations

TL;DR

This paper introduces a novel, efficient method for learning disentangled causal representations by leveraging models' generalization abilities, significantly reducing sample complexity and computation time compared to existing approaches.

Contribution

The proposed approach efficiently learns disentangled causal representations using generalization ability, avoiding reliance on adaptation speed, and is supported by theoretical analysis and empirical validation.

Findings

Achieves 1.9--11.0× more sample efficiency

Runs 9.4--32.4× faster than previous methods

Effective across various tasks

Abstract

This paper proposes an efficient approach to learning disentangled representations with causal mechanisms based on the difference of conditional probabilities in original and new distributions. We approximate the difference with models' generalization abilities so that it fits in the standard machine learning framework and can be efficiently computed. In contrast to the state-of-the-art approach, which relies on the learner's adaptation speed to new distribution, the proposed approach only requires evaluating the model's generalization ability. We provide a theoretical explanation for the advantage of the proposed method, and our experiments show that the proposed technique is 1.9--11.0$\times$ more sample efficient and 9.4--32.4 times quicker than the previous method on various tasks. The source code is available at \url{https://github.com/yuanpeng16/EDCR}.