Disentangling Node Attributes from Graph Topology for Improved Generalizability in Link Prediction

TL;DR

This paper introduces UPNA, a method that leverages pre-trained node attributes to improve the generalization and accuracy of link prediction models, especially in graphs with power-law degree distributions.

Contribution

The paper proposes UPNA, a novel unsupervised pre-training approach that disentangles node attributes from topology, enhancing link prediction and graph generation capabilities.

Findings

UPNA surpasses state-of-the-art methods by 3X to 34X on benchmark datasets.

Pre-trained node attributes improve generalization to unobserved nodes.

UPNA can be integrated with existing models for better performance.

Abstract

Link prediction is a crucial task in graph machine learning with diverse applications. We explore the interplay between node attributes and graph topology and demonstrate that incorporating pre-trained node attributes improves the generalization power of link prediction models. Our proposed method, UPNA (Unsupervised Pre-training of Node Attributes), solves the inductive link prediction problem by learning a function that takes a pair of node attributes and predicts the probability of an edge, as opposed to Graph Neural Networks (GNN), which can be prone to topological shortcuts in graphs with power-law degree distribution. In this manner, UPNA learns a significant part of the latent graph generation mechanism since the learned function can be used to add incoming nodes to a growing graph. By leveraging pre-trained node attributes, we overcome observational bias and make meaningful predictions about unobserved nodes, surpassing state-of-the-art performance (3X to 34X improvement on benchmark datasets). UPNA can be applied to various pairwise learning tasks and integrated with existing link prediction models to enhance their generalizability and bolster graph generative models.