Knowledge Graph Completion by Intermediate Variables Regularization

TL;DR

This paper introduces a novel regularization technique for tensor decomposition-based models in knowledge graph completion, which minimizes intermediate variables' norms to reduce overfitting and improve performance.

Contribution

It proposes a new regularization method applicable to TDB models that promotes low trace norm, supported by theoretical analysis and experimental validation.

Findings

Regularization improves KGC accuracy.

Theoretical analysis confirms reduced overfitting.

Method is applicable to most TDB models.

Abstract

Knowledge graph completion (KGC) can be framed as a 3-order binary tensor completion task. Tensor decomposition-based (TDB) models have demonstrated strong performance in KGC. In this paper, we provide a summary of existing TDB models and derive a general form for them, serving as a foundation for further exploration of TDB models. Despite the expressiveness of TDB models, they are prone to overfitting. Existing regularization methods merely minimize the norms of embeddings to regularize the model, leading to suboptimal performance. Therefore, we propose a novel regularization method for TDB models that addresses this limitation. The regularization is applicable to most TDB models and ensures tractable computation. Our method minimizes the norms of intermediate variables involved in the different ways of computing the predicted tensor. To support our regularization method, we provide a theoretical analysis that proves its effect in promoting low trace norm of the predicted tensor to reduce overfitting. Finally, we conduct experiments to verify the effectiveness of our regularization technique as well as the reliability of our theoretical analysis. The code is available at https://github.com/changyi7231/IVR.