G2S: A General-to-Specific Learning Framework for Temporal Knowledge Graph Forecasting with Large Language Models

TL;DR

This paper introduces G2S, a framework that disentangles general and scenario-specific knowledge learning in LLMs for temporal knowledge graph forecasting, improving their generalization capabilities.

Contribution

The paper proposes a novel G2S framework that separates general pattern learning from scenario-specific learning in LLMs for TKG forecasting.

Findings

G2S enhances LLMs' generalization in TKG forecasting.

Disentangling knowledge improves forecasting accuracy.

Framework effective across different scenarios.

Abstract

Forecasting over Temporal Knowledge Graphs (TKGs) which predicts future facts based on historical ones has received much attention. Recent studies have introduced Large Language Models (LLMs) for this task to enhance the models' generalization abilities. However, these models perform forecasting via simultaneously learning two kinds of entangled knowledge in the TKG: (1) general patterns, i.e., invariant temporal structures shared across different scenarios; and (2) scenario information, i.e., factual knowledge engaged in specific scenario, such as entities and relations. As a result, the learning processes of these two kinds of knowledge may interfere with each other, which potentially impact the generalization abilities of the models. To enhance the generalization ability of LLMs on this task, in this paper, we propose a General-to-Specific learning framework (G2S) that disentangles the learning processes of the above two kinds of knowledge. In the general learning stage, we mask the scenario information in different TKGs and convert it into anonymous temporal structures. After training on these structures, the model is able to capture the general patterns across different TKGs. In the specific learning stage, we inject the scenario information into the structures via either in-context learning or fine-tuning modes. Experimental results show that G2S effectively improves the generalization abilities of LLMs.