Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning

TL;DR

This paper introduces a hierarchical topology isomorphism expertise into graph contrastive learning, enhancing its ability to recognize graph topology and improving performance on real-world benchmarks.

Contribution

It proposes a novel plug-and-play method that incorporates hierarchical topology expertise into GCL via knowledge distillation, applicable to multiple state-of-the-art models.

Findings

Outperforms existing GCL methods on real-world benchmarks.

Achieves 0.23% and 0.43% improvements in unsupervised and transfer learning.

Provides theoretical proof of tighter Bayes error bounds.

Abstract

Graph contrastive learning (GCL) aims to align the positive features while differentiating the negative features in the latent space by minimizing a pair-wise contrastive loss. As the embodiment of an outstanding discriminative unsupervised graph representation learning approach, GCL achieves impressive successes in various graph benchmarks. However, such an approach falls short of recognizing the topology isomorphism of graphs, resulting in that graphs with relatively homogeneous node features cannot be sufficiently discriminated. By revisiting classic graph topology recognition works, we disclose that the corresponding expertise intuitively complements GCL methods. To this end, we propose a novel hierarchical topology isomorphism expertise embedded graph contrastive learning, which introduces knowledge distillations to empower GCL models to learn the hierarchical topology isomorphism expertise, including the graph-tier and subgraph-tier. On top of this, the proposed method holds the feature of plug-and-play, and we empirically demonstrate that the proposed method is universal to multiple state-of-the-art GCL models. The solid theoretical analyses are further provided to prove that compared with conventional GCL methods, our method acquires the tighter upper bound of Bayes classification error. We conduct extensive experiments on real-world benchmarks to exhibit the performance superiority of our method over candidate GCL methods, e.g., for the real-world graph representation learning experiments, the proposed method beats the state-of-the-art method by 0.23% on unsupervised representation learning setting, 0.43% on transfer learning setting. Our code is available at https://github.com/jyf123/HTML.