Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models

TL;DR

This paper introduces MEFA, a zero-shot framework utilizing multi-source evidence fuzzy aggregation with LLMs for event causality identification, effectively reducing hallucinations and outperforming unsupervised baselines.

Contribution

The paper presents a novel zero-shot approach that decomposes causality reasoning into multiple tasks and employs fuzzy aggregation to improve accuracy and reduce errors in LLM-based ECI.

Findings

MEFA outperforms second-best baselines by 6.2% in F1-score.

MEFA achieves a 9.3% improvement in precision.

The approach significantly reduces hallucination-induced errors.

Abstract

Event Causality Identification (ECI) aims to detect causal relationships between events in textual contexts. Existing ECI models predominantly rely on supervised methodologies, suffering from dependence on large-scale annotated data. Although Large Language Models (LLMs) enable zero-shot ECI, they are prone to causal hallucination-erroneously establishing spurious causal links. To address these challenges, we propose MEFA, a novel zero-shot framework based on Multi-source Evidence Fuzzy Aggregation. First, we decompose causality reasoning into three main tasks (temporality determination, necessity analysis, and sufficiency verification) complemented by three auxiliary tasks. Second, leveraging meticulously designed prompts, we guide LLMs to generate uncertain responses and deterministic outputs. Finally, we quantify LLM's responses of sub-tasks and employ fuzzy aggregation to integrate these evidence for causality scoring and causality determination. Extensive experiments on three benchmarks demonstrate that MEFA outperforms second-best unsupervised baselines by 6.2% in F1-score and 9.3% in precision, while significantly reducing hallucination-induced errors. In-depth analysis verify the effectiveness of task decomposition and the superiority of fuzzy aggregation.