Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation

TL;DR

Bridge is a novel framework that effectively combines structural knowledge graph information with semantic insights from pre-trained language models, improving triple completion accuracy.

Contribution

It introduces a joint encoding method for entities and relations using PLMs and employs a self-supervised learning approach to bridge the gap between KGs and PLMs.

Findings

Outperforms state-of-the-art models on benchmark datasets

Effectively combines structural and semantic information

Uses a novel view creation method to avoid semantic alteration

Abstract

Knowledge graph completion (KGC) is a task of inferring missing triples based on existing Knowledge Graphs (KGs). Both structural and semantic information are vital for successful KGC. However, existing methods only use either the structural knowledge from the KG embeddings or the semantic information from pre-trained language models (PLMs), leading to suboptimal model performance. Moreover, since PLMs are not trained on KGs, directly using PLMs to encode triples may be inappropriate. To overcome these limitations, we propose a novel framework called Bridge, which jointly encodes structural and semantic information of KGs. Specifically, we strategically encode entities and relations separately by PLMs to better utilize the semantic knowledge of PLMs and enable structured representation learning via a structural learning principle. Furthermore, to bridge the gap between KGs and PLMs, we employ a self-supervised representation learning method called BYOL to fine-tune PLMs with two different views of a triple. Unlike BYOL, which uses augmentation methods to create two semantically similar views of the same image, potentially altering the semantic information. We strategically separate the triple into two parts to create different views, thus avoiding semantic alteration. Experiments demonstrate that Bridge outperforms the SOTA models on three benchmark datasets.