Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning

TL;DR

This paper presents MERRY, a foundation model that integrates structural and textual information for comprehensive knowledge graph reasoning, excelling in both in-KG and out-of-KG tasks.

Contribution

Introduction of MERRY, a novel foundation model with multi-perspective encoding and dynamic fusion for enhanced reasoning and generalization in knowledge graphs.

Findings

Outperforms baselines on 28 datasets

Effective in in-KG reasoning tasks

Strong generalization to out-of-KG tasks like KGQA

Abstract

In natural language processing (NLP) and computer vision (CV), the successful application of foundation models across diverse tasks has demonstrated their remarkable potential. However, despite the rich structural and textual information embedded in knowledge graphs (KGs), existing research of foundation model for KG has primarily focused on their structural aspects, with most efforts restricted to in-KG tasks (e.g., knowledge graph completion, KGC). This limitation has hindered progress in addressing more challenging out-of-KG tasks. In this paper, we introduce MERRY, a foundation model for general knowledge graph reasoning, and investigate its performance across two task categories: in-KG reasoning tasks (e.g., KGC) and out-of-KG tasks (e.g., KG question answering, KGQA). We not only utilize the structural information, but also the textual information in KGs. Specifically, we propose a multi-perspective Conditional Message Passing (CMP) encoding architecture to bridge the gap between textual and structural modalities, enabling their seamless integration. Additionally, we introduce a dynamic residual fusion module to selectively retain relevant textual information and a flexible edge scoring mechanism to adapt to diverse downstream tasks. Comprehensive evaluations on 28 datasets demonstrate that MERRY outperforms existing baselines in most scenarios, showcasing strong reasoning capabilities within KGs and excellent generalization to out-of-KG tasks such as KGQA.