Enhanced Language Representation with Label Knowledge for Span Extraction

TL;DR

This paper introduces a new method for span extraction that explicitly integrates label knowledge into text representations, achieving state-of-the-art results while significantly reducing training and inference times.

Contribution

The paper proposes a novel model that efficiently incorporates label knowledge into span extraction, outperforming QA-based methods in both accuracy and speed.

Findings

Achieves state-of-the-art performance on four benchmarks.

Reduces training time by 76% and inference time by 77%.

Effective across flat NER, nested NER, and event detection tasks.

Abstract

Span extraction, aiming to extract text spans (such as words or phrases) from plain texts, is a fundamental process in Information Extraction. Recent works introduce the label knowledge to enhance the text representation by formalizing the span extraction task into a question answering problem (QA Formalization), which achieves state-of-the-art performance. However, QA Formalization does not fully exploit the label knowledge and suffers from low efficiency in training/inference. To address those problems, we introduce a new paradigm to integrate label knowledge and further propose a novel model to explicitly and efficiently integrate label knowledge into text representations. Specifically, it encodes texts and label annotations independently and then integrates label knowledge into text representation with an elaborate-designed semantics fusion module. We conduct extensive experiments on three typical span extraction tasks: flat NER, nested NER, and event detection. The empirical results show that 1) our method achieves state-of-the-art performance on four benchmarks, and 2) reduces training time and inference time by 76% and 77% on average, respectively, compared with the QA Formalization paradigm. Our code and data are available at https://github.com/Akeepers/LEAR.