A Multimodal Text- and Graph-Based Approach for Open-Domain Event Extraction from Documents

TL;DR

This paper introduces MODEE, a multimodal approach combining graph-based learning and LLMs for open-domain event extraction, improving accuracy and generalization over existing methods.

Contribution

The paper presents a novel multimodal framework that effectively models document-level reasoning for open-domain event extraction, outperforming prior approaches.

Findings

MODEE outperforms state-of-the-art open-domain event extraction methods.

MODEE generalizes well to closed-domain event extraction.

Empirical results demonstrate superior performance on large datasets.

Abstract

Event extraction is essential for event understanding and analysis. It supports tasks such as document summarization and decision-making in emergency scenarios. However, existing event extraction approaches have limitations: (1) closed-domain algorithms are restricted to predefined event types and thus rarely generalize to unseen types and (2) open-domain event extraction algorithms, capable of handling unconstrained event types, have largely overlooked the potential of large language models (LLMs) despite their advanced abilities. Additionally, they do not explicitly model document-level contextual, structural, and semantic reasoning, which are crucial for effective event extraction but remain challenging for LLMs due to lost-in-the-middle phenomenon and attention dilution. To address these limitations, we propose multimodal open-domain event extraction, MODEE , a novel approach for open-domain event extraction that combines graph-based learning with text-based representation from LLMs to model document-level reasoning. Empirical evaluations on large datasets demonstrate that MODEE outperforms state-of-the-art open-domain event extraction approaches and can be generalized to closed-domain event extraction, where it outperforms existing algorithms.