Design Challenges and Misconceptions in Neural Sequence Labeling

TL;DR

This paper systematically compares twelve neural sequence labeling models across three benchmarks, clarifies misconceptions in the literature, and offers practical insights for designing effective neural sequence labeling systems.

Contribution

It reproduces and compares most state-of-the-art neural sequence labeling models, clarifies misconceptions, and provides practical guidelines for practitioners.

Findings

Most models perform similarly on benchmarks.

Certain misconceptions in literature are clarified.

Practical recommendations for model design.

Abstract

We investigate the design challenges of constructing effective and efficient neural sequence labeling systems, by reproducing twelve neural sequence labeling models, which include most of the state-of-the-art structures, and conduct a systematic model comparison on three benchmarks (i.e. NER, Chunking, and POS tagging). Misconceptions and inconsistent conclusions in existing literature are examined and clarified under statistical experiments. In the comparison and analysis process, we reach several practical conclusions which can be useful to practitioners.