A Contextualized BERT model for Knowledge Graph Completion

TL;DR

This paper introduces a contextualized BERT model for Knowledge Graph Completion that leverages neighboring entity and relationship context, eliminating the need for entity descriptions and negative sampling, and achieves state-of-the-art results.

Contribution

The proposed model utilizes contextual information from neighboring nodes to improve KGC without relying on entity descriptions or negative sampling, reducing computational costs.

Findings

Outperforms state-of-the-art on FB15k-237 and WN18RR datasets.

Improves Hit@1 by 5.3% and 4.88% respectively.

Reduces computational demands compared to textual-based models.

Abstract

Knowledge graphs (KGs) are valuable for representing structured, interconnected information across domains, enabling tasks like semantic search, recommendation systems and inference. A pertinent challenge with KGs, however, is that many entities (i.e., heads, tails) or relationships are unknown. Knowledge Graph Completion (KGC) addresses this by predicting these missing nodes or links, enhancing the graph's informational depth and utility. Traditional methods like TransE and ComplEx predict tail entities but struggle with unseen entities. Textual-based models leverage additional semantics but come with high computational costs, semantic inconsistencies, and data imbalance issues. Recent LLM-based models show improvement but overlook contextual information and rely heavily on entity descriptions. In this study, we introduce a contextualized BERT model for KGC that overcomes these limitations by utilizing the contextual information from neighbouring entities and relationships to predict tail entities. Our model eliminates the need for entity descriptions and negative triplet sampling, reducing computational demands while improving performance. Our model outperforms state-of-the-art methods on standard datasets, improving Hit@1 by 5.3% and 4.88% on FB15k-237 and WN18RR respectively, setting a new benchmark in KGC.