Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs

TL;DR

This paper presents a schema-aware event extraction method using LLMs that decomposes tasks and incorporates dynamic retrieval to improve accuracy and reduce hallucination, outperforming baseline approaches.

Contribution

It introduces a novel approach combining task decomposition and dynamic schema-aware retrieval to enhance LLM-based event extraction accuracy.

Findings

Superior performance on event extraction benchmarks

Effective reduction of hallucination in LLM outputs

Enhanced adaptability through dynamic retrieval examples

Abstract

Large Language Models (LLMs) demonstrate significant capabilities in processing natural language data, promising efficient knowledge extraction from diverse textual sources to enhance situational awareness and support decision-making. However, concerns arise due to their susceptibility to hallucination, resulting in contextually inaccurate content. This work focuses on harnessing LLMs for automated Event Extraction, introducing a new method to address hallucination by decomposing the task into Event Detection and Event Argument Extraction. Moreover, the proposed method integrates dynamic schema-aware augmented retrieval examples into prompts tailored for each specific inquiry, thereby extending and adapting advanced prompting techniques such as Retrieval-Augmented Generation. Evaluation findings on prominent event extraction benchmarks and results from a synthesized benchmark illustrate the method's superior performance compared to baseline approaches.