Structure-Aware Decoding Mechanisms for Complex Entity Extraction with Large-Scale Language Models

TL;DR

This paper introduces a structure-aware decoding approach for large language models that improves nested and overlapping entity extraction by modeling hierarchical relationships and structural constraints, leading to higher accuracy.

Contribution

The paper presents a novel structure-aware decoding mechanism that jointly models entity boundaries, hierarchies, and dependencies, enhancing extraction performance in complex scenarios.

Findings

Significant improvements in Accuracy, Precision, Recall, and F1-Score on ACE 2005 dataset.

Enhanced boundary localization and structural modeling in nested and overlapping entities.

Effective in maintaining semantic and structural consistency in complex extractions.

Abstract

This paper proposes a structure-aware decoding method based on large language models to address the difficulty of traditional approaches in maintaining both semantic integrity and structural consistency in nested and overlapping entity extraction tasks. The method introduces a candidate span generation mechanism and structured attention modeling to achieve unified modeling of entity boundaries, hierarchical relationships, and cross-dependencies. The model first uses a pretrained language model to obtain context-aware semantic representations, then captures multi-granular entity span features through candidate representation combinations, and introduces hierarchical structural constraints during decoding to ensure consistency between semantics and structure. To enhance stability in complex scenarios, the model jointly optimizes classification loss and structural consistency loss, maintaining high recognition accuracy under multi-entity co-occurrence and long-sentence dependency conditions. Experiments conducted on the ACE 2005 dataset demonstrate significant improvements in Accuracy, Precision, Recall, and F1-Score, particularly in nested and overlapping entity recognition, where the model shows stronger boundary localization and structural modeling capability. This study verifies the effectiveness of structure-aware decoding in complex semantic extraction tasks, provides a new perspective for developing language models with hierarchical understanding, and establishes a methodological foundation for high-precision information extraction.