Towards Foundation Model on Temporal Knowledge Graph Reasoning

TL;DR

This paper introduces POSTRA, a fully-inductive, scalable foundation model for temporal knowledge graph reasoning that generalizes to unseen entities, relations, and timestamps using sinusoidal encodings and message passing.

Contribution

It presents the first fully-inductive approach for temporal knowledge graph link prediction, enabling zero-shot transfer and generalization across diverse temporal graphs.

Findings

POSTRA achieves strong zero-shot performance on unseen TKGs.

The model effectively generalizes to new entities, relations, and timestamps.

The approach is agnostic to temporal granularity and span.

Abstract

Temporal Knowledge Graphs (TKGs) store temporal facts with quadruple formats (s, p, o, t). Existing Temporal Knowledge Graph Embedding (TKGE) models perform link prediction tasks in transductive or semi-inductive settings, which means the entities, relations, and temporal information in the test graph are fully or partially observed during training. Such reliance on seen elements during inference limits the models' ability to transfer to new domains and generalize to real-world scenarios. A central limitation is the difficulty in learning representations for entities, relations, and timestamps that are transferable and not tied to dataset-specific vocabularies. To overcome these limitations, we introduce the first fully-inductive approach to temporal knowledge graph link prediction. Our model employs sinusoidal positional encodings to capture fine-grained temporal patterns and generates adaptive entity and relation representations using message passing conditioned on both local and global temporal contexts. Our model design is agnostic to temporal granularity and time span, effectively addressing temporal discrepancies across TKGs and facilitating time-aware structural information transfer. As a pretrained, scalable, and transferable model, POSTRA demonstrates strong zero-shot performance on unseen temporal knowledge graphs, effectively generalizing to novel entities, relations, and timestamps. Extensive theoretical analysis and empirical results show that a single pretrained model can improve zero-shot performance on various inductive temporal reasoning scenarios, marking a significant step toward a foundation model for temporal KGs.