Toward Better Temporal Structures for Geopolitical Events Forecasting

TL;DR

This paper introduces HTKGH, a generalized framework for geopolitical event forecasting that enhances expressiveness over traditional hyper-relational temporal knowledge graphs, and evaluates large language models on this new dataset.

Contribution

It formalizes HTKGH, creates the htkgh-polecat dataset, and benchmarks LLMs, demonstrating improved expressiveness and insights into LLM capabilities for complex temporal forecasting.

Findings

HTKGH formalization improves knowledge graph expressiveness.

LLMs show varying performance on complex geopolitical forecasting.

The new dataset enables better evaluation of forecasting models.

Abstract

Forecasting on geopolitical temporal knowledge graphs (TKGs) through the lens of large language models (LLMs) has recently gained traction. While TKGs and their generalization, hyper-relational temporal knowledge graphs (HTKGs), offer a straightforward structure to represent simple temporal relationships, they lack the expressive power to convey complex facts efficiently. One of the critical limitations of HTKGs is a lack of support for more than two primary entities in temporal facts, which commonly occur in real-world events. To address this limitation, in this work, we study a generalization of HTKGs, Hyper-Relational Temporal Knowledge Generalized Hypergraphs (HTKGHs). We first derive a formalization for HTKGHs, demonstrating their backward compatibility while supporting two complex types of facts commonly found in geopolitical incidents. Then, utilizing this formalization, we introduce the htkgh-polecat dataset, built upon the global event database POLECAT. Finally, we benchmark and analyze popular LLMs on our dataset, providing insights into 1) the positive impact of utilizing the HTKGH formalization compared to existing ones and 2) LLMs' adaptability and capabilities in complex forecasting tasks.