Mixture of Link Predictors on Graphs

TL;DR

This paper introduces Link-MoE, a mixture of experts model for link prediction in graphs that dynamically selects the best pairwise information source for each node pair, significantly improving prediction accuracy.

Contribution

It proposes a novel mixture of experts approach that leverages multiple GNNs for different pairwise information types, addressing the limitations of uniform information application in link prediction.

Findings

Achieves an 18.71% relative improvement in MRR on Pubmed.

Attains a 9.59% relative improvement in Hits@100 on ogbl-ppa.

Demonstrates substantial performance gains across diverse real-world datasets.

Abstract

Link prediction, which aims to forecast unseen connections in graphs, is a fundamental task in graph machine learning. Heuristic methods, leveraging a range of different pairwise measures such as common neighbors and shortest paths, often rival the performance of vanilla Graph Neural Networks (GNNs). Therefore, recent advancements in GNNs for link prediction (GNN4LP) have primarily focused on integrating one or a few types of pairwise information. In this work, we reveal that different node pairs within the same dataset necessitate varied pairwise information for accurate prediction and models that only apply the same pairwise information uniformly could achieve suboptimal performance. As a result, we propose a simple mixture of experts model Link-MoE for link prediction. Link-MoE utilizes various GNNs as experts and strategically selects the appropriate expert for each node pair based on various types of pairwise information. Experimental results across diverse real-world datasets demonstrate substantial performance improvement from Link-MoE. Notably, Link-MoE achieves a relative improvement of 18.71\% on the MRR metric for the Pubmed dataset and 9.59\% on the Hits@100 metric for the ogbl-ppa dataset, compared to the best baselines.