A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation

TL;DR

This paper proposes a unified framework linking causal and traditional representation learning, fostering mutual insights and practical improvements through experimental analysis of task and constraint interactions.

Contribution

It introduces a unified formulation for representation learning combining task and constraint components, bridging causal and traditional paradigms.

Findings

Causal constraints' effectiveness varies with task types.

Structured latent constraints influence CRL performance.

Experimental results on CausalVerse demonstrate task-dependent benefits.

Abstract

Causal representation learning (CRL) and traditional representation learning have largely developed along different trajectories. Traditional representation learning has been driven mainly by applications and empirical objectives, whereas CRL has focused more on theoretical questions, particularly identifiability. This difference in emphasis has created a gap between the two fields in terminology, problem formulation, and evaluation, limiting communication and sometimes leading to disconnected or redundant efforts. In this paper, we argue that these two fields should be brought into dialogue rather than treated as separate paradigms. To this end, we introduce a unified formulation in which the representation learning is characterized by two components: a task component, which specifies what information the learned representation is required to preserve, and a constraint component, which specifies what structure is imposed on the latent space. Under this formulation, the benefits run in both directions. CRL provides theoretical tools for understanding when structured latent constraints are useful or necessary, while traditional representation learning offers practical insights on task design and objective choice that can improve the development of CRL methods. To illustrate this interaction, we experimentally study how different task components affect the behavior of CRL methods under different structured constraints. Results on CausalVerse show that the effectiveness of causal constraints depends strongly on the tasks with which they are paired.