Temporal Smoothness Regularisers for Neural Link Predictors

TL;DR

This paper investigates how different temporal smoothing regularisers affect neural link predictors for dynamic knowledge graphs, demonstrating that careful regulariser selection can significantly improve temporal link prediction accuracy.

Contribution

It systematically analyzes various temporal regularisers for neural link predictors, showing that simple models can achieve state-of-the-art results with appropriate regularisation.

Findings

Careful regulariser selection improves prediction accuracy

Simple tensor factorisation models can outperform complex models with proper regularisation

Proposed regularisers lead to new state-of-the-art results on temporal link prediction datasets

Abstract

Most algorithms for representation learning and link prediction on relational data are designed for static data. However, the data to which they are applied typically evolves over time, including online social networks or interactions between users and items in recommender systems. This is also the case for graph-structured knowledge bases -- knowledge graphs -- which contain facts that are valid only for specific points in time. In such contexts, it becomes crucial to correctly identify missing links at a precise time point, i.e. the temporal prediction link task. Recently, Lacroix et al. and Sadeghian et al. proposed a solution to the problem of link prediction for knowledge graphs under temporal constraints inspired by the canonical decomposition of 4-order tensors, where they regularise the representations of time steps by enforcing temporal smoothing, i.e. by learning similar transformation for adjacent timestamps. However, the impact of the choice of temporal regularisation terms is still poorly understood. In this work, we systematically analyse several choices of temporal smoothing regularisers using linear functions and recurrent architectures. In our experiments, we show that by carefully selecting the temporal smoothing regulariser and regularisation weight, a simple method like TNTComplEx can produce significantly more accurate results than state-of-the-art methods on three widely used temporal link prediction datasets. Furthermore, we evaluate the impact of a wide range of temporal smoothing regularisers on two state-of-the-art temporal link prediction models. Our work shows that simple tensor factorisation models can produce new state-of-the-art results using newly proposed temporal regularisers, highlighting a promising avenue for future research.