MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning

TL;DR

This paper introduces MA-GCL, a novel graph contrastive learning paradigm that manipulates view encoder architectures with three tricks—asymmetric, random, and shuffling—to improve diversity and performance in graph representation learning.

Contribution

The paper proposes a new model augmentation approach for GCL that focuses on encoder architecture manipulation, achieving state-of-the-art results.

Findings

MA-GCL achieves state-of-the-art node classification performance.

The three augmentation tricks effectively improve view diversity and model robustness.

Extensive experiments validate the effectiveness of each trick.

Abstract

Contrastive learning (CL), which can extract the information shared between different contrastive views, has become a popular paradigm for vision representation learning. Inspired by the success in computer vision, recent work introduces CL into graph modeling, dubbed as graph contrastive learning (GCL). However, generating contrastive views in graphs is more challenging than that in images, since we have little prior knowledge on how to significantly augment a graph without changing its labels. We argue that typical data augmentation techniques (e.g., edge dropping) in GCL cannot generate diverse enough contrastive views to filter out noises. Moreover, previous GCL methods employ two view encoders with exactly the same neural architecture and tied parameters, which further harms the diversity of augmented views. To address this limitation, we propose a novel paradigm named model augmented GCL (MA-GCL), which will focus on manipulating the architectures of view encoders instead of perturbing graph inputs. Specifically, we present three easy-to-implement model augmentation tricks for GCL, namely asymmetric, random and shuffling, which can respectively help alleviate high- frequency noises, enrich training instances and bring safer augmentations. All three tricks are compatible with typical data augmentations. Experimental results show that MA-GCL can achieve state-of-the-art performance on node classification benchmarks by applying the three tricks on a simple base model. Extensive studies also validate our motivation and the effectiveness of each trick. (Code, data and appendix are available at https://github.com/GXM1141/MA-GCL. )