Graph Entropy Minimization for Semi-supervised Node Classification

TL;DR

This paper introduces Graph Entropy Minimization (GEM), a semi-supervised learning method that improves node classification accuracy, reduces training resources, and accelerates inference in graph neural networks, suitable for resource-constrained industrial applications.

Contribution

GEM is a novel semi-supervised learning approach that combines one-hop aggregation with stochastic training and online knowledge distillation for efficient, accurate, and fast node classification.

Findings

GEM achieves accuracy comparable to multi-hop GNNs with only one-hop aggregation.

GEM enables fast, space-efficient training using mini-batches of edge samples.

GEM's inference speed surpasses deep GNNs, especially with online knowledge distillation.

Abstract

Node classifiers are required to comprehensively reduce prediction errors, training resources, and inference latency in the industry. However, most graph neural networks (GNN) concentrate only on one or two of them. The compromised aspects thus are the shortest boards on the bucket, hindering their practical deployments for industrial-level tasks. This work proposes a novel semi-supervised learning method termed Graph Entropy Minimization (GEM) to resolve the three issues simultaneously. GEM benefits its one-hop aggregation from massive uncategorized nodes, making its prediction accuracy comparable to GNNs with two or more hops message passing. It can be decomposed to support stochastic training with mini-batches of independent edge samples, achieving extremely fast sampling and space-saving training. While its one-hop aggregation is faster in inference than deep GNNs, GEM can be further accelerated to an extreme by deriving a non-hop classifier via online knowledge distillation. Thus, GEM can be a handy choice for latency-restricted and error-sensitive services running on resource-constraint hardware. Code is available at https://github.com/cf020031308/GEM.