Semi-supervised Training for Knowledge Base Graph Self-attention Networks on Link Prediction

TL;DR

This paper introduces a semi-supervised self-training approach combined with a redesigned self-attention mechanism for GCN-based models, significantly improving link prediction accuracy on knowledge graphs.

Contribution

It proposes a novel semi-supervised self-training method and a redesigned self-attention mechanism to enhance GCN-based link prediction models.

Findings

Improved Hits@1 by about 30% on FB15k-237

Effective enhancement of link prediction performance

Validated on benchmark datasets FB15k-237 and WN18RR

Abstract

The task of link prediction aims to solve the problem of incomplete knowledge caused by the difficulty of collecting facts from the real world. GCNs-based models are widely applied to solve link prediction problems due to their sophistication, but GCNs-based models are suffering from two problems in the structure and training process. 1) The transformation methods of GCN layers become increasingly complex in GCN-based knowledge representation models; 2) Due to the incompleteness of the knowledge graph collection process, there are many uncollected true facts in the labeled negative samples. Therefore, this paper investigates the characteristic of the information aggregation coefficient (self-attention) of adjacent nodes and redesigns the self-attention mechanism of the GAT structure. Meanwhile, inspired by human thinking habits, we designed a semi-supervised self-training method over pre-trained models. Experimental results on the benchmark datasets FB15k-237 and WN18RR show that our proposed self-attention mechanism and semi-supervised self-training method can effectively improve the performance of the link prediction task. If you look at FB15k-237, for example, the proposed method improves Hits@1 by about 30%.