Relation-First Modeling Paradigm for Causal Representation Learning toward the Development of AGI

TL;DR

This paper proposes a relation-first modeling paradigm and the Relation-Indexed Representation Learning (RIRL) method to improve causal representation learning by focusing on causal relations rather than traditional i.i.d. assumptions.

Contribution

It introduces a novel relation-first perspective for causality and presents RIRL as a practical approach validated by experiments.

Findings

RIRL effectively captures causal relations in experiments.

The relation-first paradigm addresses limitations of traditional i.i.d. methods.

Experimental results demonstrate improved causal representation learning.

Abstract

The traditional i.i.d.-based learning paradigm faces inherent challenges in addressing causal relationships, which has become increasingly evident with the rise of applications in causal representation learning. Our understanding of causality naturally requires a perspective as the creator rather than observer, as the ``what...if'' questions only hold within the possible world we conceive. The traditional perspective limits capturing dynamic causal outcomes and leads to compensatory efforts such as the reliance on hidden confounders. This paper lays the groundwork for the new perspective, which enables the \emph{relation-first} modeling paradigm for causality. Also, it introduces the Relation-Indexed Representation Learning (RIRL) as a practical implementation, supported by experiments that validate its efficacy.