OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding

TL;DR

OmniEvent is a versatile toolkit that supports multiple event understanding tasks across various datasets, ensuring fair evaluation and user-friendly deployment for researchers and practitioners.

Contribution

It introduces a comprehensive, fair, and easy-to-use toolkit for event understanding, supporting multiple paradigms and datasets with off-the-shelf models and modular design.

Findings

Supports 15 datasets in English and Chinese

Addresses evaluation pitfalls for fair comparison

Provides deployable web service models

Abstract

Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction. To facilitate related research and application, we present an event understanding toolkit OmniEvent, which features three desiderata: (1) Comprehensive. OmniEvent supports mainstream modeling paradigms of all the event understanding tasks and the processing of 15 widely-used English and Chinese datasets. (2) Fair. OmniEvent carefully handles the inconspicuous evaluation pitfalls reported in Peng et al. (2023), which ensures fair comparisons between different models. (3) Easy-to-use. OmniEvent is designed to be easily used by users with varying needs. We provide off-the-shelf models that can be directly deployed as web services. The modular framework also enables users to easily implement and evaluate new event understanding models with OmniEvent. The toolkit (https://github.com/THU-KEG/OmniEvent) is publicly released along with the demonstration website and video (https://omnievent.xlore.cn/).