Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling

TL;DR

This paper introduces WORLDREP, a high-quality dataset for predicting international events from text, created using large-language models and expert validation to improve research in geopolitics and policy decision-making.

Contribution

The paper presents a novel, rigorously validated dataset for text-based event prediction, enhancing data quality and supporting advanced research in international event forecasting.

Findings

WORLDREP improves event prediction accuracy

The dataset is validated by domain experts

Open-source tools facilitate future research

Abstract

Predicting future international events from textual information, such as news articles, has tremendous potential for applications in global policy, strategic decision-making, and geopolitics. However, existing datasets available for this task are often limited in quality, hindering the progress of related research. In this paper, we introduce WORLDREP (WORLD Relationship and Event Prediction), a novel dataset designed to address these limitations by leveraging the advanced reasoning capabilities of large-language models (LLMs). Our dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. We showcase the quality and utility of WORLDREP for real-world event prediction tasks, demonstrating its effectiveness through extensive experiments and analysis. Furthermore, we publicly release our dataset along with the full automation source code for data collection, labeling, and benchmarking, aiming to support and advance research in text-based event prediction.