NOTE: Solution for KDD-CUP 2021 WikiKG90M-LSC

TL;DR

This paper presents an ensemble approach combining advanced knowledge graph embedding methods, including a modified OTE, DeepWalk, and statistical features, to improve triplet prediction in the large WikiKG90M knowledge graph for KDD Cup 2021.

Contribution

It introduces NOTE, a modified version of OTE, and demonstrates how combining multiple embedding methods and feature engineering enhances triplet prediction accuracy.

Findings

Ensemble of methods outperforms individual models.

Modified OTE (NOTE) improves embedding quality.

Feature engineering boosts prediction performance.

Abstract

WikiKG90M in KDD Cup 2021 is a large encyclopedic knowledge graph, which could benefit various downstream applications such as question answering and recommender systems. Participants are invited to complete the knowledge graph by predicting missing triplets. Recent representation learning methods have achieved great success on standard datasets like FB15k-237. Thus, we train the advanced algorithms in different domains to learn the triplets, including OTE, QuatE, RotatE and TransE. Significantly, we modified OTE into NOTE (short for Norm-OTE) for better performance. Besides, we use both the DeepWalk and the post-smoothing technique to capture the graph structure for supplementation. In addition to the representations, we also use various statistical probabilities among the head entities, the relations and the tail entities for the final prediction. Experimental results show that the ensemble of state-of-the-art representation learning methods could draw on each others strengths. And we develop feature engineering from validation candidates for further improvements. Please note that we apply the same strategy on the test set for final inference. And these features may not be practical in the real world when considering ranking against all the entities.