Neural CRF transducers for sequence labeling

TL;DR

This paper introduces NCRF transducers, a novel sequence labeling model combining two RNNs to capture complex dependencies, leading to improved performance over existing neural CRF models in tasks like POS tagging, chunking, and NER.

Contribution

The paper proposes NCRF transducers, a new neural sequence labeling model that captures long-range label dependencies using two RNNs, advancing beyond traditional linear-chain NCRFs.

Findings

Achieves consistent improvements over linear-chain NCRFs.

Outperforms RNN transducers on multiple sequence labeling tasks.

Improves state-of-the-art results in POS tagging, chunking, and NER.

Abstract

Conditional random fields (CRFs) have been shown to be one of the most successful approaches to sequence labeling. Various linear-chain neural CRFs (NCRFs) are developed to implement the non-linear node potentials in CRFs, but still keeping the linear-chain hidden structure. In this paper, we propose NCRF transducers, which consists of two RNNs, one extracting features from observations and the other capturing (theoretically infinite) long-range dependencies between labels. Different sequence labeling methods are evaluated over POS tagging, chunking and NER (English, Dutch). Experiment results show that NCRF transducers achieve consistent improvements over linear-chain NCRFs and RNN transducers across all the four tasks, and can improve state-of-the-art results.