Adaptive Schema-aware Event Extraction with Retrieval-Augmented Generation

TL;DR

This paper introduces ASEE, a novel event extraction method that combines schema paraphrasing with retrieval-augmented generation, addressing schema rigidity and lack of benchmarks, and demonstrates improved accuracy across diverse datasets.

Contribution

The paper presents ASEE, a new schema-aware event extraction paradigm that enhances adaptability and accuracy, along with the MD-SEE benchmark for comprehensive evaluation.

Findings

ASEE outperforms existing methods on MD-SEE benchmark.

ASEE effectively retrieves and paraphrases schemas for better extraction.

The benchmark consolidates 12 datasets across multiple domains.

Abstract

Event extraction (EE) is a fundamental task in natural language processing (NLP) that involves identifying and extracting event information from unstructured text. Effective EE in real-world scenarios requires two key steps: selecting appropriate schemas from hundreds of candidates and executing the extraction process. Existing research exhibits two critical gaps: (1) the rigid schema fixation in existing pipeline systems, and (2) the absence of benchmarks for evaluating joint schema matching and extraction. Although large language models (LLMs) offer potential solutions, their schema hallucination tendencies and context window limitations pose challenges for practical deployment. In response, we propose Adaptive Schema-aware Event Extraction (ASEE), a novel paradigm combining schema paraphrasing with schema retrieval-augmented generation. ASEE adeptly retrieves paraphrased schemas and accurately generates targeted structures. To facilitate rigorous evaluation, we construct the Multi-Dimensional Schema-aware Event Extraction (MD-SEE) benchmark, which systematically consolidates 12 datasets across diverse domains, complexity levels, and language settings. Extensive evaluations on MD-SEE show that our proposed ASEE demonstrates strong adaptability across various scenarios, significantly improving the accuracy of event extraction.