A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation

TL;DR

This paper introduces a scalable pretraining framework for link prediction that combines module transferability, a Mixture-of-Experts approach, and efficient tuning, achieving state-of-the-art results with minimal computational cost.

Contribution

It presents the first systematic study on transferability of pairwise modules, a late fusion strategy, and a Mixture-of-Experts framework for efficient, adaptable link prediction pretraining.

Findings

Achieves state-of-the-art performance on low-resource datasets.

Over 10,000x reduction in computational overhead compared to end-to-end training.

Effective across 16 diverse datasets.

Abstract

Link Prediction (LP) is a critical task in graph machine learning. While Graph Neural Networks (GNNs) have significantly advanced LP performance recently, existing methods face key challenges including limited supervision from sparse connectivity, sensitivity to initialization, and poor generalization under distribution shifts. We explore pretraining as a solution to address these challenges. Unlike node classification, LP is inherently a pairwise task, which requires the integration of both node- and edge-level information. In this work, we present the first systematic study on the transferability of these distinct modules and propose a late fusion strategy to effectively combine their outputs for improved performance. To handle the diversity of pretraining data and avoid negative transfer, we introduce a Mixture-of-Experts (MoE) framework that captures distinct patterns in separate experts, facilitating seamless application of the pretrained model on diverse downstream datasets. For fast adaptation, we develop a parameter-efficient tuning strategy that allows the pretrained model to adapt to unseen datasets with minimal computational overhead. Experiments on 16 datasets across two domains demonstrate the effectiveness of our approach, achieving state-of-the-art performance on low-resource link prediction while obtaining competitive results compared to end-to-end trained methods, with over 10,000x lower computational overhead.