RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU

TL;DR

RexUniNLU is a universal NLU framework that unifies information extraction and text classification tasks using explicit schema constraints, covering a wide range of schemas and modalities with improved accuracy.

Contribution

The paper introduces a formal universal information extraction formulation and a recursive method with explicit schema constraints for unified NLU tasks, including IE and CLS across multiple schemas and modalities.

Findings

Effective across IE and CLS tasks in English and Chinese

Outperforms existing models in accuracy and versatility

Successfully handles complex schemas like quadruples and quintuples

Abstract

Information Extraction (IE) and Text Classification (CLS) serve as the fundamental pillars of NLU, with both disciplines relying on analyzing input sequences to categorize outputs into pre-established schemas. However, there is no existing encoder-based model that can unify IE and CLS tasks from this perspective. To fully explore the foundation shared within NLU tasks, we have proposed a Recursive Method with Explicit Schema Instructor for Universal NLU. Specifically, we firstly redefine the true universal information extraction (UIE) with a formal formulation that covers almost all extraction schemas, including quadruples and quintuples which remain unsolved for previous UIE models. Then, we expands the formulation to all CLS and multi-modal NLU tasks. Based on that, we introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS, which encompasses all IE and CLS tasks and prevent incorrect connections between schema and input sequence. To avoid interference between different schemas, we reset the position ids and attention mask matrices. Extensive experiments are conducted on IE, CLS in both English and Chinese, and multi-modality, revealing the effectiveness and superiority. Our codes are publicly released.