Transfer Entropy in Graph Convolutional Neural Networks

TL;DR

This paper introduces a Transfer Entropy-based method to address oversmoothing and heterophily in Graph Convolutional Networks, improving accuracy but increasing computational cost.

Contribution

It proposes a novel TE-based strategy to enhance GCN performance by leveraging node relational properties and information transfer measures.

Findings

TE-based node selection improves classification accuracy

Using heterophily and degree info enhances GCN performance

Significant computational overhead when applying TE to many nodes

Abstract

Graph Convolutional Networks (GCN) are Graph Neural Networks where the convolutions are applied over a graph. In contrast to Convolutional Neural Networks, GCN's are designed to perform inference on graphs, where the number of nodes can vary, and the nodes are unordered. In this study, we address two important challenges related to GCNs: i) oversmoothing; and ii) the utilization of node relational properties (i.e., heterophily and homophily). Oversmoothing is the degradation of the discriminative capacity of nodes as a result of repeated aggregations. Heterophily is the tendency for nodes of different classes to connect, whereas homophily is the tendency of similar nodes to connect. We propose a new strategy for addressing these challenges in GCNs based on Transfer Entropy (TE), which measures of the amount of directed transfer of information between two time varying nodes. Our findings indicate that using node heterophily and degree information as a node selection mechanism, along with feature-based TE calculations, enhances accuracy across various GCN models. Our model can be easily modified to improve classification accuracy of a GCN model. As a trade off, this performance boost comes with a significant computational overhead when the TE is computed for many graph nodes.