Document-Level Event Extraction with Definition-Driven ICL

TL;DR

This paper introduces a definition-driven optimization strategy for document-level event extraction using LLMs, improving performance by refining prompts, balancing data, and enhancing heuristic clarity.

Contribution

It presents a novel DDEE approach that addresses prompt design challenges and enhances extraction accuracy in LLM-based document-level event extraction.

Findings

Improved event extraction accuracy with optimized prompts.

Enhanced model generalization through data balancing.

Increased precision via structured heuristics and strict conditions.

Abstract

In the field of Natural Language Processing (NLP), Large Language Models (LLMs) have shown great potential in document-level event extraction tasks, but existing methods face challenges in the design of prompts. To address this issue, we propose an optimization strategy called "Definition-driven Document-level Event Extraction (DDEE)." By adjusting the length of the prompt and enhancing the clarity of heuristics, we have significantly improved the event extraction performance of LLMs. We used data balancing techniques to solve the long-tail effect problem, enhancing the model's generalization ability for event types. At the same time, we refined the prompt to ensure it is both concise and comprehensive, adapting to the sensitivity of LLMs to the style of prompts. In addition, the introduction of structured heuristic methods and strict limiting conditions has improved the precision of event and argument role extraction. These strategies not only solve the prompt engineering problems of LLMs in document-level event extraction but also promote the development of event extraction technology, providing new research perspectives for other tasks in the NLP field.