SSL Framework for Causal Inconsistency between Structures and Representations
Hang Chen, Xinyu Yang, Keqing Du, Wenya Wang

TL;DR
This paper introduces a self-supervised learning framework to address causal inconsistency in complex data types, enhancing causal structure and representation learning in deep learning models.
Contribution
It identifies causal inconsistency in Indefinite Data and proposes a novel SSL framework based on intervention to improve causal alignment.
Findings
SSL framework improves causal consistency
Enhances causal structure and representation learning
Benefits extend to downstream tasks and LLM instructions
Abstract
The cross-pollination between causal discovery and deep learning has led to increasingly extensive interactions. It results in a large number of deep learning data types (such as images, text, etc.) extending into the field of causal discovery, and a multitude of deep learning tasks have begun to utilize causal discovery to explore the internal causal structure and causal representation of data. In this paper, we first identified that a complex data type, ``Indefinite Data", has conflicts between causal relationships expressed by the causal structure and causal representation generated by deep learning models, a phenomenon referred to as causal inconsistency. We thoroughly analyzed related work to explain why only Indefinite Data exhibits causal inconsistency while other data types do not. Furthermore, to alleviate causal inconsistency, we proposed a self-supervised learning (SSL)…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Advanced Graph Neural Networks
