Generating Harder Cross-document Event Coreference Resolution Datasets using Metaphoric Paraphrasing

TL;DR

This paper introduces ECB+META, a challenging new dataset for cross-document event coreference resolution that incorporates lexical diversity and figurative language, using ChatGPT for metaphoric sentence transformation.

Contribution

The authors create ECB+META, a lexically diverse and figuratively rich dataset for CDEC, leveraging ChatGPT for metaphoric paraphrasing without re-annotating coreference links.

Findings

Existing methods perform poorly on ECB+META compared to ECB+

ECB+META presents a more challenging benchmark for CDEC models

Metaphoric language significantly impacts coreference resolution performance

Abstract

The most popular Cross-Document Event Coreference Resolution (CDEC) datasets fail to convey the true difficulty of the task, due to the lack of lexical diversity between coreferring event triggers (words or phrases that refer to an event). Furthermore, there is a dearth of event datasets for figurative language, limiting a crucial avenue of research in event comprehension. We address these two issues by introducing ECB+META, a lexically rich variant of Event Coref Bank Plus (ECB+) for CDEC on symbolic and metaphoric language. We use ChatGPT as a tool for the metaphoric transformation of sentences in the documents of ECB+, then tag the original event triggers in the transformed sentences in a semi-automated manner. In this way, we avoid the re-annotation of expensive coreference links. We present results that show existing methods that work well on ECB+ struggle with ECB+META, thereby paving the way for CDEC research on a much more challenging dataset. Code/data: https://github.com/ahmeshaf/llms_coref