Heterogeneous Graph Neural Networks with Loss-decrease-aware Curriculum Learning

TL;DR

This paper introduces LDHGNN, a novel training approach for heterogeneous graph neural networks that leverages loss decrease trends to improve learning efficiency and address data imbalance, demonstrating enhanced performance.

Contribution

The paper proposes a loss-decrease-aware curriculum learning method for HGNNs, utilizing loss trend analysis and a sampling strategy to improve training effectiveness.

Findings

Enhanced HGNN performance on downstream tasks.

Effective handling of training imbalance issues.

Demonstrated superiority over existing methods.

Abstract

In recent years, heterogeneous graph neural networks (HGNNs) have achieved excellent performance in handling heterogeneous information networks (HINs). Curriculum learning is a machine learning strategy where training examples are presented to a model in a structured order, starting with easy examples and gradually increasing difficulty, aiming to improve learning efficiency and generalization. To better exploit the rich information in HINs, previous methods have started to explore the use of curriculum learning strategy to train HGNNs. Specifically, these works utilize the absolute value of the loss at each training epoch to evaluate the learning difficulty of each training sample. However, the relative loss, rather than the absolute value of loss, reveals the learning difficulty. Therefore, we propose a novel loss-decrease-aware training schedule (LDTS). LDTS uses the trend of loss decrease between each training epoch to better evaluating the difficulty of training samples, thereby enhancing the curriculum learning of HGNNs for downstream tasks. Additionally, we propose a sampling strategy to alleviate training imbalance issues. Our method further demonstrate the efficacy of curriculum learning in enhancing HGNNs capabilities. We call our method Loss-decrease-aware Heterogeneous Graph Neural Networks (LDHGNN). The code is public at https://github.com/wangyili00/LDHGNN.