Causal Graph based Event Reasoning using Semantic Relation Experts

TL;DR

This paper introduces a method for generating causal event graphs using semantic relation experts to improve event reasoning in LLMs, enhancing explainability and performance on forecasting and prediction tasks.

Contribution

The paper proposes a collaborative expert-based approach for causal graph generation and demonstrates its utility in improving event reasoning and explainability without fine-tuning.

Findings

Causal graphs improve event reasoning accuracy.

Expert discussion rounds enhance graph quality.

Method achieves competitive results on forecasting tasks.

Abstract

Understanding how events in a scenario causally connect with each other is important for effectively modeling and reasoning about events. But event reasoning remains a difficult challenge, and despite recent advances, Large Language Models (LLMs) still struggle to accurately identify causal connections between events. This struggle leads to poor performance on deeper reasoning tasks like event forecasting and timeline understanding. To address this challenge, we investigate the generation of causal event graphs (e.g., A enables B) as a parallel mechanism to help LLMs explicitly represent causality during inference. This paper evaluates both how to generate correct graphs as well as how graphs can assist reasoning. We propose a collaborative approach to causal graph generation where we use LLMs to simulate experts that focus on specific semantic relations. The experts engage in multiple rounds of discussions which are then consolidated by a final expert. Then, to demonstrate the utility of causal graphs, we use them on multiple downstream applications, and also introduce a new explainable event prediction task that requires a causal chain of events in the explanation. These explanations are more informative and coherent than baseline generations. Finally, our overall approach not finetuned on any downstream task, achieves competitive results with state-of-the-art models on both forecasting and next event prediction tasks.