DiP-GNN: Discriminative Pre-Training of Graph Neural Networks

TL;DR

DiP-GNN introduces a discriminative pre-training framework for GNNs that improves graph matching by training a generator and discriminator to recover masked edges, enhancing downstream task performance.

Contribution

The paper proposes a novel discriminative pre-training method for GNNs that addresses graph mismatch issues in traditional generative approaches.

Findings

Effective on large-scale homogeneous graphs

Improves node classification accuracy

Outperforms existing pre-training methods

Abstract

Graph neural network (GNN) pre-training methods have been proposed to enhance the power of GNNs. Specifically, a GNN is first pre-trained on a large-scale unlabeled graph and then fine-tuned on a separate small labeled graph for downstream applications, such as node classification. One popular pre-training method is to mask out a proportion of the edges, and a GNN is trained to recover them. However, such a generative method suffers from graph mismatch. That is, the masked graph inputted to the GNN deviates from the original graph. To alleviate this issue, we propose DiP-GNN (Discriminative Pre-training of Graph Neural Networks). Specifically, we train a generator to recover identities of the masked edges, and simultaneously, we train a discriminator to distinguish the generated edges from the original graph's edges. In our framework, the graph seen by the discriminator better matches the original graph because the generator can recover a proportion of the masked edges. Extensive experiments on large-scale homogeneous and heterogeneous graphs demonstrate the effectiveness of the proposed framework.