Norm Augmented Graph AutoEncoders for Link Prediction

TL;DR

This paper identifies degree-related bias in Graph AutoEncoders for link prediction and proposes a norm augmentation method with self-loops to improve performance on low-degree nodes, showing significant empirical gains.

Contribution

The study reveals the impact of embedding norms on LP performance and introduces a simple, effective norm augmentation technique to mitigate degree bias in GAEs.

Findings

Norm of node embeddings varies with node degree.

Increasing embedding norms improves LP accuracy for low-degree nodes.

Norm-augmented GAEs outperform standard methods across datasets.

Abstract

Link Prediction (LP) is a crucial problem in graph-structured data. Graph Neural Networks (GNNs) have gained prominence in LP, with Graph AutoEncoders (GAEs) being a notable representation. However, our empirical findings reveal that GAEs' LP performance suffers heavily from the long-tailed node degree distribution, i.e., low-degree nodes tend to exhibit inferior LP performance compared to high-degree nodes. \emph{What causes this degree-related bias, and how can it be mitigated?} In this study, we demonstrate that the norm of node embeddings learned by GAEs exhibits variation among nodes with different degrees, underscoring its central significance in influencing the final performance of LP. Specifically, embeddings with larger norms tend to guide the decoder towards predicting higher scores for positive links and lower scores for negative links, thereby contributing to superior performance. This observation motivates us to improve GAEs' LP performance on low-degree nodes by increasing their embedding norms, which can be implemented simply yet effectively by introducing additional self-loops into the training objective for low-degree nodes. This norm augmentation strategy can be seamlessly integrated into existing GAE methods with light computational cost. Extensive experiments on various datasets and GAE methods show the superior performance of norm-augmented GAEs.