SEMMA: A Semantic Aware Knowledge Graph Foundation Model

TL;DR

SEMMA is a novel knowledge graph foundation model that combines textual semantics and structural information, significantly improving zero-shot reasoning and generalization in diverse and unseen graph scenarios.

Contribution

SEMMA introduces a dual-module approach that integrates textual semantics from LLMs with structural data, enhancing transferability and zero-shot reasoning in knowledge graphs.

Findings

SEMMA outperforms structural baselines like ULTRA in inductive link prediction.

In unseen relation settings, SEMMA is twice as effective as purely structural methods.

Textual semantics are essential for generalization when structural cues are insufficient.

Abstract

Knowledge Graph Foundation Models (KGFMs) have shown promise in enabling zero-shot reasoning over unseen graphs by learning transferable patterns. However, most existing KGFMs rely solely on graph structure, overlooking the rich semantic signals encoded in textual attributes. We introduce SEMMA, a dual-module KGFM that systematically integrates transferable textual semantics alongside structure. SEMMA leverages Large Language Models (LLMs) to enrich relation identifiers, generating semantic embeddings that subsequently form a textual relation graph, which is fused with the structural component. Across 54 diverse KGs, SEMMA outperforms purely structural baselines like ULTRA in fully inductive link prediction. Crucially, we show that in more challenging generalization settings, where the test-time relation vocabulary is entirely unseen, structural methods collapse while SEMMA is 2x more effective. Our findings demonstrate that textual semantics are critical for generalization in settings where structure alone fails, highlighting the need for foundation models that unify structural and linguistic signals in knowledge reasoning.