Hierarchically-Refined Label Attention Network for Sequence Labeling

TL;DR

This paper introduces a hierarchically-refined label attention network that enhances sequence labeling by explicitly leveraging label embeddings and capturing long-term dependencies, resulting in improved accuracy and efficiency.

Contribution

The proposed model advances sequence labeling by integrating hierarchical attention and label embeddings, outperforming BiLSTM-CRF in accuracy and speed.

Findings

Improved tagging accuracy on POS, NER, and CCG tasks.

Faster training and testing compared to BiLSTM-CRF.

Effective modeling of long-term label dependencies.

Abstract

CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.