History repeats Itself: A Baseline for Temporal Knowledge Graph Forecasting

TL;DR

This paper introduces a simple, effective baseline for Temporal Knowledge Graph Forecasting based on recurring facts, revealing that it often outperforms complex models and highlighting the need for proper evaluation standards.

Contribution

The paper presents a straightforward baseline for TKG forecasting that requires minimal tuning and training, challenging assumptions about the superiority of complex models.

Findings

Baseline ranks first or third on three datasets

Simple baseline outperforms many complex models

Highlights importance of proper evaluation protocols

Abstract

Temporal Knowledge Graph (TKG) Forecasting aims at predicting links in Knowledge Graphs for future timesteps based on a history of Knowledge Graphs. To this day, standardized evaluation protocols and rigorous comparison across TKG models are available, but the importance of simple baselines is often neglected in the evaluation, which prevents researchers from discerning actual and fictitious progress. We propose to close this gap by designing an intuitive baseline for TKG Forecasting based on predicting recurring facts. Compared to most TKG models, it requires little hyperparameter tuning and no iterative training. Further, it can help to identify failure modes in existing approaches. The empirical findings are quite unexpected: compared to 11 methods on five datasets, our baseline ranks first or third in three of them, painting a radically different picture of the predictive quality of the state of the art.