TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Models

TL;DR

TFMLinker introduces a universal link prediction method leveraging tabular foundation models' in-context learning, capturing topological information without dataset-specific fine-tuning, and demonstrating superior performance across diverse graph benchmarks.

Contribution

The paper proposes TFMLinker, a novel approach that adapts tabular foundation models for universal link prediction in graphs, overcoming previous limitations of GNNs and LLMs.

Findings

Outperforms state-of-the-art baselines on 6 graph benchmarks.

Does not require dataset-specific fine-tuning.

Effectively captures topological information for link prediction.

Abstract

Link prediction is a fundamental task in graph machine learning with widespread applications such as recommendation systems, drug discovery, knowledge graphs, etc. In the foundation model era, how to develop universal link prediction methods across datasets and domains becomes a key problem, with some initial attempts adopting Graph Foundation Models utilizing Graph Neural Networks and Large Language Models. However, the existing methods face notable limitations, including limited pre-training scale or heavy reliance on textual information. Motivated by the success of tabular foundation models (TFMs) in achieving universal prediction across diverse tabular datasets, we explore an alternative approach by TFMs, which are pre-trained on diverse synthetic datasets sampled from structural causal models and support strong in-context learning independent of textual attributes. Nevertheless, adapting TFMs for link prediction faces severe technical challenges such as how to obtain the necessary context and capture link-centric topological information. To solve these challenges, we propose TFMLinker (Tabular Foundation Model for Link Predictor), aiming to leverage the in-context learning capabilities of TFMs to perform link prediction across diverse graphs without requiring dataset-specific fine-tuning. Specifically, we first develop a prototype-augmented local-global context module to construct context that captures both graph-specific and cross-graph transferable patterns. Next, we design a universal topology-aware link encoder to capture link-centric topological information and generate link representations as inputs for the TFM. Finally, we employ the TFM to predict link existence through in-context learning. Experiments on 6 graph benchmarks across diverse domains demonstrate the superiority of our method over state-of-the-art baselines without requiring dataset-specific finetuning.