How Expressive are Knowledge Graph Foundation Models?

TL;DR

This paper investigates the expressive power of Knowledge Graph Foundation Models, revealing that their ability to represent knowledge depends on the motifs used, and demonstrates that richer motifs enhance their performance.

Contribution

It provides a theoretical analysis linking motifs to expressiveness in KGFMs and introduces more expressive models using complex motifs, validated by empirical results.

Findings

Richer motifs increase model expressiveness.

Binary motifs limit the models' ability to capture complex relations.

Using complex motifs improves performance across diverse datasets.

Abstract

Knowledge Graph Foundation Models (KGFMs) are at the frontier for deep learning on knowledge graphs (KGs), as they can generalize to completely novel knowledge graphs with different relational vocabularies. Despite their empirical success, our theoretical understanding of KGFMs remains very limited. In this paper, we conduct a rigorous study of the expressive power of KGFMs. Specifically, we show that the expressive power of KGFMs directly depends on the motifs that are used to learn the relation representations. We then observe that the most typical motifs used in the existing literature are binary, as the representations are learned based on how pairs of relations interact, which limits the model's expressiveness. As part of our study, we design more expressive KGFMs using richer motifs, which necessitate learning relation representations based on, e.g., how triples of relations interact with each other. Finally, we empirically validate our theoretical findings, showing that the use of richer motifs results in better performance on a wide range of datasets drawn from different domains.