Adaptive Multi-layer Contrastive Graph Neural Networks

TL;DR

This paper introduces AMC-GNN, a self-supervised graph neural network framework that adaptively learns feature representations across multiple layers using contrastive learning and attention mechanisms, improving performance on various graph benchmarks.

Contribution

The paper proposes AMC-GNN, a novel multi-layer contrastive learning framework with adaptive layer importance weighting and an auxiliary encoder for enhanced graph representation learning.

Findings

Consistent accuracy improvements across multiple graph benchmarks.

Effective use of attention mechanism for layer importance weighting.

Enhanced representation quality demonstrated on new datasets.

Abstract

We present Adaptive Multi-layer Contrastive Graph Neural Networks (AMC-GNN), a self-supervised learning framework for Graph Neural Network, which learns feature representations of sample data without data labels. AMC-GNN generates two graph views by data augmentation and compares different layers' output embeddings of Graph Neural Network encoders to obtain feature representations, which could be used for downstream tasks. AMC-GNN could learn the importance weights of embeddings in different layers adaptively through the attention mechanism, and an auxiliary encoder is introduced to train graph contrastive encoders better. The accuracy is improved by maximizing the representation's consistency of positive pairs in the early layers and the final embedding space. Our experiments show that the results can be consistently improved by using the AMC-GNN framework, across four established graph benchmarks: Cora, Citeseer, Pubmed, DBLP citation network datasets, as well as four newly proposed datasets: Co-author-CS, Co-author-Physics, Amazon-Computers, Amazon-Photo.