LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning

TL;DR

This paper introduces a novel LLM-assisted knowledge distillation method tailored for temporal knowledge graph reasoning, enhancing lightweight models' performance by leveraging large language models as auxiliary teachers.

Contribution

It presents a specialized distillation framework that incorporates LLMs to improve temporal reasoning in compact TKG models, addressing limitations of static graph distillation techniques.

Findings

Consistently outperforms strong distillation baselines on multiple benchmarks.

Enables lightweight models to better capture event dynamics without increased inference costs.

Demonstrates the effectiveness of LLMs as teachers for temporal reasoning transfer.

Abstract

Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.