Locally-Contextual Nonlinear CRFs for Sequence Labeling

TL;DR

This paper introduces locally-contextual nonlinear CRFs that leverage neighboring embeddings directly, improving sequence labeling performance over traditional linear CRFs and achieving state-of-the-art results on key benchmarks.

Contribution

The paper proposes a novel nonlinear CRF architecture that incorporates local contextual information via neural networks, enhancing sequence labeling accuracy.

Findings

Outperforms linear CRFs in ablation studies

Achieves state-of-the-art on CoNLL 2000 chunking

Sets new benchmarks on OntoNotes NER

Abstract

Linear chain conditional random fields (CRFs) combined with contextual word embeddings have achieved state of the art performance on sequence labeling tasks. In many of these tasks, the identity of the neighboring words is often the most useful contextual information when predicting the label of a given word. However, contextual embeddings are usually trained in a task-agnostic manner. This means that although they may encode information about the neighboring words, it is not guaranteed. It can therefore be beneficial to design the sequence labeling architecture to directly extract this information from the embeddings. We propose locally-contextual nonlinear CRFs for sequence labeling. Our approach directly incorporates information from the neighboring embeddings when predicting the label for a given word, and parametrizes the potential functions using deep neural networks. Our model serves as a drop-in replacement for the linear chain CRF, consistently outperforming it in our ablation study. On a variety of tasks, our results are competitive with those of the best published methods. In particular, we outperform the previous state of the art on chunking on CoNLL 2000 and named entity recognition on OntoNotes 5.0 English.