Learning from the Dark: Boosting Graph Convolutional Neural Networks with Diverse Negative Samples

TL;DR

This paper introduces a method to enhance Graph Convolutional Neural Networks by incorporating diverse negative samples from the graph's dark world, improving performance and reducing over-smoothing.

Contribution

It proposes a novel negative sampling strategy using determinantal point processes to incorporate negative information into GCNs, which is a new approach in graph learning.

Findings

Improved node representation learning performance.

Significant reduction in over-smoothing effects.

Effective negative sampling strategy enhances GCNs.

Abstract

Graph Convolutional Neural Networks (GCNs) has been generally accepted to be an effective tool for node representations learning. An interesting way to understand GCNs is to think of them as a message passing mechanism where each node updates its representation by accepting information from its neighbours (also known as positive samples). However, beyond these neighbouring nodes, graphs have a large, dark, all-but forgotten world in which we find the non-neighbouring nodes (negative samples). In this paper, we show that this great dark world holds a substantial amount of information that might be useful for representation learning. Most specifically, it can provide negative information about the node representations. Our overall idea is to select appropriate negative samples for each node and incorporate the negative information contained in these samples into the representation updates. Moreover, we show that the process of selecting the negative samples is not trivial. Our theme therefore begins by describing the criteria for a good negative sample, followed by a determinantal point process algorithm for efficiently obtaining such samples. A GCN, boosted by diverse negative samples, then jointly considers the positive and negative information when passing messages. Experimental evaluations show that this idea not only improves the overall performance of standard representation learning but also significantly alleviates over-smoothing problems.