EventFull: Complete and Consistent Event Relation Annotation

TL;DR

EventFull is a novel annotation tool that enables efficient, consistent, and complete annotation of various event relations, improving dataset quality for NLP tasks.

Contribution

It introduces a unified tool for systematic annotation of event relations, addressing the challenge of quadratic pairwise annotation complexity.

Findings

Accelerates annotation process

Achieves high inter-annotator agreement

Supports comprehensive event relation annotation

Abstract

Event relation detection is a fundamental NLP task, leveraged in many downstream applications, whose modeling requires datasets annotated with event relations of various types. However, systematic and complete annotation of these relations is costly and challenging, due to the quadratic number of event pairs that need to be considered. Consequently, many current event relation datasets lack systematicity and completeness. In response, we introduce \textit{EventFull}, the first tool that supports consistent, complete and efficient annotation of temporal, causal and coreference relations via a unified and synergetic process. A pilot study demonstrates that EventFull accelerates and simplifies the annotation process while yielding high inter-annotator agreement.