SpeechEE: A Novel Benchmark for Speech Event Extraction

TL;DR

This paper introduces SpeechEE, a new benchmark for extracting event information directly from speech signals, including a large dataset and a novel end-to-end model with innovative modules, setting a foundation for future research.

Contribution

It pioneers the SpeechEE task, creates a large-scale speech event extraction dataset, and proposes a novel end-to-end model with unique modules for effective speech event extraction.

Findings

The proposed model outperforms baselines on SpeechEE datasets.

The dataset covers diverse scenarios, languages, and speaker styles.

SpeechEE provides a strong baseline for future research in speech event extraction.

Abstract

Event extraction (EE) is a critical direction in the field of information extraction, laying an important foundation for the construction of structured knowledge bases. EE from text has received ample research and attention for years, yet there can be numerous real-world applications that require direct information acquisition from speech signals, online meeting minutes, interview summaries, press releases, etc. While EE from speech has remained under-explored, this paper fills the gap by pioneering a SpeechEE, defined as detecting the event predicates and arguments from a given audio speech. To benchmark the SpeechEE task, we first construct a large-scale high-quality dataset. Based on textual EE datasets under the sentence, document, and dialogue scenarios, we convert texts into speeches through both manual real-person narration and automatic synthesis, empowering the data with diverse scenarios, languages, domains, ambiences, and speaker styles. Further, to effectively address the key challenges in the task, we tailor an E2E SpeechEE system based on the encoder-decoder architecture, where a novel Shrinking Unit module and a retrieval-aided decoding mechanism are devised. Extensive experimental results on all SpeechEE subsets demonstrate the efficacy of the proposed model, offering a strong baseline for the task. At last, being the first work on this topic, we shed light on key directions for future research.