Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models

TL;DR

This paper introduces a novel knowledge distillation framework that leverages large language models to improve temporal knowledge graph reasoning, focusing on efficiency and temporal dependency preservation for resource-constrained applications.

Contribution

The paper presents a tailored distillation method for TKG reasoning that effectively transfers temporal and structural reasoning from large language models to lightweight student models.

Findings

Outperforms existing baselines on benchmark datasets

Achieves a good balance between accuracy and efficiency

Enhances reasoning capabilities with temporal dynamics

Abstract

Reasoning over temporal knowledge graphs (TKGs) is fundamental to improving the efficiency and reliability of intelligent decision-making systems and has become a key technological foundation for future artificial intelligence applications. Despite recent progress, existing TKG reasoning models typically rely on large parameter sizes and intensive computation, leading to high hardware costs and energy consumption. These constraints hinder their deployment on resource-constrained, low-power, and distributed platforms that require real-time inference. Moreover, most existing model compression and distillation techniques are designed for static knowledge graphs and fail to adequately capture the temporal dependencies inherent in TKGs, often resulting in degraded reasoning performance. To address these challenges, we propose a distillation framework specifically tailored for temporal knowledge graph reasoning. Our approach leverages large language models as teacher models to guide the distillation process, enabling effective transfer of both structural and temporal reasoning capabilities to lightweight student models. By integrating large-scale public knowledge with task-specific temporal information, the proposed framework enhances the student model's ability to model temporal dynamics while maintaining a compact and efficient architecture. Extensive experiments on multiple publicly available benchmark datasets demonstrate that our method consistently outperforms strong baselines, achieving a favorable trade-off between reasoning accuracy, computational efficiency, and practical deployability.