Graph Contrastive Invariant Learning from the Causal Perspective

TL;DR

This paper analyzes graph contrastive learning from a causal perspective, identifying issues with non-causal information, and proposes a spectral augmentation and invariance objectives to improve learning of causal, invariant representations.

Contribution

It introduces a causal perspective to GCL, proposing spectral augmentation and invariance objectives to enhance causal invariant representation learning.

Findings

Improved node classification accuracy

Effective reduction of non-causal influence

Better invariant representation learning

Abstract

Graph contrastive learning (GCL), learning the node representation by contrasting two augmented graphs in a self-supervised way, has attracted considerable attention. GCL is usually believed to learn the invariant representation. However, does this understanding always hold in practice? In this paper, we first study GCL from the perspective of causality. By analyzing GCL with the structural causal model (SCM), we discover that traditional GCL may not well learn the invariant representations due to the non-causal information contained in the graph. How can we fix it and encourage the current GCL to learn better invariant representations? The SCM offers two requirements and motives us to propose a novel GCL method. Particularly, we introduce the spectral graph augmentation to simulate the intervention upon non-causal factors. Then we design the invariance objective and independence objective to better capture the causal factors. Specifically, (i) the invariance objective encourages the encoder to capture the invariant information contained in causal variables, and (ii) the independence objective aims to reduce the influence of confounders on the causal variables. Experimental results demonstrate the effectiveness of our approach on node classification tasks.