ReGrAt: Regularization in Graphs using Attention to handle class imbalance

TL;DR

This paper introduces ReGrAt, a novel regularization approach using attention mechanisms to address class imbalance in graph node classification, achieving state-of-the-art results without resampling data.

Contribution

The work proposes a new regularization method leveraging attention networks to mitigate class imbalance in graph-based node classification tasks.

Findings

Achieves state-of-the-art performance on citation benchmark datasets.

Using attention-based regularization improves minority class recognition.

Implicitly handles imbalance without resampling or data augmentation.

Abstract

Node classification is an important task to solve in graph-based learning. Even though a lot of work has been done in this field, imbalance is neglected. Real-world data is not perfect, and is imbalanced in representations most of the times. Apart from text and images, data can be represented using graphs, and thus addressing the imbalance in graphs has become of paramount importance. In the context of node classification, one class has less examples than others. Changing data composition is a popular way to address the imbalance in node classification. This is done by resampling the data to balance the dataset. However, that can sometimes lead to loss of information or add noise to the dataset. Therefore, in this work, we implicitly solve the problem by changing the model loss. Specifically, we study how attention networks can help tackle imbalance. Moreover, we observe that using a regularizer to assign larger weights to minority nodes helps to mitigate this imbalance. We achieve State of the Art results than the existing methods on several standard citation benchmark datasets.