Rethinking Regularization Methods for Knowledge Graph Completion

TL;DR

This paper rethinks regularization in knowledge graph completion, demonstrating that well-designed regularization enhances performance and introduces a novel sparse-regularization method that outperforms existing techniques.

Contribution

It offers a new perspective on regularization in KGC and proposes a rank-based sparse regularizer that improves model performance beyond previous limits.

Findings

Regularization alleviates overfitting and reduces variance.

The proposed SPR regularizer outperforms other regularization methods.

SPR enables models to surpass previous performance bounds.

Abstract

Knowledge graph completion (KGC) has attracted considerable attention in recent years because it is critical to improving the quality of knowledge graphs. Researchers have continuously explored various models. However, most previous efforts have neglected to take advantage of regularization from a deeper perspective and therefore have not been used to their full potential. This paper rethinks the application of regularization methods in KGC. Through extensive empirical studies on various KGC models, we find that carefully designed regularization not only alleviates overfitting and reduces variance but also enables these models to break through the upper bounds of their original performance. Furthermore, we introduce a novel sparse-regularization method that embeds the concept of rank-based selective sparsity into the KGC regularizer. The core idea is to selectively penalize those components with significant features in the embedding vector, thus effectively ignoring many components that contribute little and may only represent noise. Various comparative experiments on multiple datasets and multiple models show that the SPR regularization method is better than other regularization methods and can enable the KGC model to further break through the performance margin.