KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning

TL;DR

KGPFN is a novel knowledge graph foundation model that leverages in-context learning and structured local and global contexts to improve reasoning across unseen graphs, outperforming fine-tuned models.

Contribution

It introduces KGPFN, combining relation graph message passing with in-context learning using a Prior-Data Fitted Network for better generalization.

Findings

Achieves strong adaptation to unseen graphs using in-context learning.

Outperforms fine-tuned models on 57 KG benchmarks.

Effectively captures relational invariances and contextual evidence.

Abstract

Knowledge graph (KG) foundation models aim to generalize across graphs with unseen entities and relations by learning transferable relational structure. However, most existing methods primarily emphasize relation-level universality, while in-context learning, the other pillar of foundation models remains under-explored for KG reasoning. In KGs, context is inherently structured and heterogeneous: effective prediction requires conditioning on the local context around the query entities as well as the global context that summarizes how a relation behaves across many instances. We propose KGPFN, a KG foundation model using Prior-data Fitted Network that unifies transferable relational regularities with inference-time in-context learning from structured context. KGPFN first learns relation representations via message passing on relation graphs to capture cross-graph relational invariances. For query-specific reasoning, it encodes local neighborhoods using a multi-layer NBFNet as local context. To enable ICL at global scale, it constructs relation-specific global context by retrieving a large set of instances of the query relation together with their local neighborhoods, and aggregates them within a Prior-Data Fitted Network framework that combines feature-level and sample-level attention. Through multi-graph pretraining on diverse KGs, KGPFN learns when to instantiate reusable patterns and when to override them using contextual evidence. Experiments on 57 KG benchmarks demonstrate that KGPFN achieves strong adaptation to previously unseen graphs through in-context learning alone, consistently outperforming competitive fine-tuned KG foundation models. Our code is available at https://github.com/HKUST-KnowComp/KGPFN.