LTE4G: Long-Tail Experts for Graph Neural Networks

TL;DR

LTE4G introduces a novel GNN training framework that jointly addresses class and degree long-tail issues by using specialized experts and knowledge distillation, significantly improving classification on imbalanced graphs.

Contribution

The paper proposes LTE4G, a new framework that considers both class and degree long-tailness in GNN training, using expert models and knowledge distillation for better performance.

Findings

LTE4G outperforms state-of-the-art methods on imbalanced graph datasets.

The expert-based approach effectively handles both class and degree long-tail distributions.

Knowledge distillation improves the specialization of class-wise classifiers.

Abstract

Existing Graph Neural Networks (GNNs) usually assume a balanced situation where both the class distribution and the node degree distribution are balanced. However, in real-world situations, we often encounter cases where a few classes (i.e., head class) dominate other classes (i.e., tail class) as well as in the node degree perspective, and thus naively applying existing GNNs eventually fall short of generalizing to the tail cases. Although recent studies proposed methods to handle long-tail situations on graphs, they only focus on either the class long-tailedness or the degree long-tailedness. In this paper, we propose a novel framework for training GNNs, called Long-Tail Experts for Graphs (LTE4G), which jointly considers the class long-tailedness, and the degree long-tailedness for node classification. The core idea is to assign an expert GNN model to each subset of nodes that are split in a balanced manner considering both the class and degree long-tailedness. After having trained an expert for each balanced subset, we adopt knowledge distillation to obtain two class-wise students, i.e., Head class student and Tail class student, each of which is responsible for classifying nodes in the head classes and tail classes, respectively. We demonstrate that LTE4G outperforms a wide range of state-of-the-art methods in node classification evaluated on both manual and natural imbalanced graphs. The source code of LTE4G can be found at https://github.com/SukwonYun/LTE4G.