Packed Levitated Marker for Entity and Relation Extraction

TL;DR

This paper introduces Packed Levitated Markers (PL-Marker), a novel span representation method that models interrelations between spans for improved entity and relation extraction, achieving state-of-the-art results.

Contribution

The paper proposes a new span representation approach with packing strategies that consider span interrelations, enhancing extraction performance.

Findings

Improves baselines on six NER benchmarks.

Achieves 4.1%-4.3% F1 improvement on ACE datasets.

Provides higher speed compared to previous models.

Abstract

Recent entity and relation extraction works focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the interrelation between spans (pairs). In this work, we propose a novel span representation approach, named Packed Levitated Markers (PL-Marker), to consider the interrelation between the spans (pairs) by strategically packing the markers in the encoder. In particular, we propose a neighborhood-oriented packing strategy, which considers the neighbor spans integrally to better model the entity boundary information. Furthermore, for those more complicated span pair classification tasks, we design a subject-oriented packing strategy, which packs each subject and all its objects to model the interrelation between the same-subject span pairs. The experimental results show that, with the enhanced marker feature, our model advances baselines on six NER benchmarks, and obtains a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models on ACE04 and ACE05.