LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model

TL;DR

This paper introduces LasUIE, a structure-aware generative language model that leverages syntactic knowledge and heterogeneous structural representations to improve universal information extraction across multiple tasks.

Contribution

It proposes a novel structure-aware GLM with a heterogeneous structure inductor, structural broadcaster, and task-oriented fine-tuning, enhancing UIE performance by incorporating syntactic structures.

Findings

Significant improvements over baseline UIE systems on 12 benchmarks

Learned task-adaptive structural bias effectively addresses long-range dependence

Enhanced boundary identification and structural understanding in IE tasks

Abstract

Universally modeling all typical information extraction tasks (UIE) with one generative language model (GLM) has revealed great potential by the latest study, where various IE predictions are unified into a linearized hierarchical expression under a GLM. Syntactic structure information, a type of effective feature which has been extensively utilized in IE community, should also be beneficial to UIE. In this work, we propose a novel structure-aware GLM, fully unleashing the power of syntactic knowledge for UIE. A heterogeneous structure inductor is explored to unsupervisedly induce rich heterogeneous structural representations by post-training an existing GLM. In particular, a structural broadcaster is devised to compact various latent trees into explicit high-order forests, helping to guide a better generation during decoding. We finally introduce a task-oriented structure fine-tuning mechanism, further adjusting the learned structures to most coincide with the end-task's need. Over 12 IE benchmarks across 7 tasks our system shows significant improvements over the baseline UIE system. Further in-depth analyses show that our GLM learns rich task-adaptive structural bias that greatly resolves the UIE crux, the long-range dependence issue and boundary identifying. Source codes are open at https://github.com/ChocoWu/LasUIE.