Let Invariant Rationale Discovery Inspire Graph Contrastive Learning

TL;DR

This paper introduces RGCL, a graph contrastive learning framework that uses a rationale generator to identify salient features, leading to improved representation learning and state-of-the-art results on various graph datasets.

Contribution

The paper proposes a novel rationale-aware graph contrastive learning framework that automatically discovers salient features without supervision, enhancing graph representation learning.

Findings

Rationale generator effectively captures salient features in graphs.

RGCL achieves state-of-the-art performance on benchmark datasets.

Visual analysis confirms the rationales focus on semantic nodes.

Abstract

Leading graph contrastive learning (GCL) methods perform graph augmentations in two fashions: (1) randomly corrupting the anchor graph, which could cause the loss of semantic information, or (2) using domain knowledge to maintain salient features, which undermines the generalization to other domains. Taking an invariance look at GCL, we argue that a high-performing augmentation should preserve the salient semantics of anchor graphs regarding instance-discrimination. To this end, we relate GCL with invariant rationale discovery, and propose a new framework, Rationale-aware Graph Contrastive Learning (RGCL). Specifically, without supervision signals, RGCL uses a rationale generator to reveal salient features about graph instance-discrimination as the rationale, and then creates rationale-aware views for contrastive learning. This rationale-aware pre-training scheme endows the backbone model with the powerful representation ability, further facilitating the fine-tuning on downstream tasks. On MNIST-Superpixel and MUTAG datasets, visual inspections on the discovered rationales showcase that the rationale generator successfully captures the salient features (i.e. distinguishing semantic nodes in graphs). On biochemical molecule and social network benchmark datasets, the state-of-the-art performance of RGCL demonstrates the effectiveness of rationale-aware views for contrastive learning. Our codes are available at https://github.com/lsh0520/RGCL.