Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering

TL;DR

This paper introduces a knowledge-guided binary question answering approach for document-level causal relation extraction, addressing challenges of modeling and hallucinations, and achieves state-of-the-art results with large language models.

Contribution

It proposes a novel two-stage method leveraging event structures and binary QA to improve document-level causal relation extraction with large language models.

Findings

Achieves state-of-the-art on MECI dataset.

Demonstrates high generalizability and low inconsistency.

Effective use of event structures enhances document-level ECRE.

Abstract

As an essential task in information extraction (IE), Event-Event Causal Relation Extraction (ECRE) aims to identify and classify the causal relationships between event mentions in natural language texts. However, existing research on ECRE has highlighted two critical challenges, including the lack of document-level modeling and causal hallucinations. In this paper, we propose a Knowledge-guided binary Question Answering (KnowQA) method with event structures for ECRE, consisting of two stages: Event Structure Construction and Binary Question Answering. We conduct extensive experiments under both zero-shot and fine-tuning settings with large language models (LLMs) on the MECI and MAVEN-ERE datasets. Experimental results demonstrate the usefulness of event structures on document-level ECRE and the effectiveness of KnowQA by achieving state-of-the-art on the MECI dataset. We observe not only the effectiveness but also the high generalizability and low inconsistency of our method, particularly when with complete event structures after fine-tuning the models.