Named Entity Recognition with stack residual LSTM and trainable bias decoding

TL;DR

This paper introduces residual connections in stacked RNNs and a bias decoding mechanism to enhance NER performance, achieving state-of-the-art results on CoNLL 2003 for English and Spanish.

Contribution

It proposes residual connections for deep RNNs and a bias decoding method to optimize non-differentiable objectives in NER models.

Findings

Improved NER accuracy on CoNLL 2003 dataset

Achieved state-of-the-art results for English and Spanish

Demonstrated effectiveness of bias decoding in NER

Abstract

Recurrent Neural Network models are the state-of-the-art for Named Entity Recognition (NER). We present two innovations to improve the performance of these models. The first innovation is the introduction of residual connections between the Stacked Recurrent Neural Network model to address the degradation problem of deep neural networks. The second innovation is a bias decoding mechanism that allows the trained system to adapt to non-differentiable and externally computed objectives, such as the entity-based F-measure. Our work improves the state-of-the-art results for both Spanish and English languages on the standard train/development/test split of the CoNLL 2003 Shared Task NER dataset.