An Empirical Evaluation of Sequence-Tagging Trainers

TL;DR

This paper empirically compares various online and batch discriminative sequence-labeling algorithms, analyzing their speed, accuracy, and convergence to guide practitioners in selecting suitable methods.

Contribution

It provides a systematic comparison of sequence-labeling algorithms, highlighting how their performance varies based on evaluation criteria and implementation complexity.

Findings

Online algorithms are faster on large datasets.

Algorithm choice depends on evaluation criteria.

Implementation simplicity influences selection.

Abstract

The task of assigning label sequences to a set of observed sequences is common in computational linguistics. Several models for sequence labeling have been proposed over the last few years. Here, we focus on discriminative models for sequence labeling. Many batch and online (updating model parameters after visiting each example) learning algorithms have been proposed in the literature. On large datasets, online algorithms are preferred as batch learning methods are slow. These online algorithms were designed to solve either a primal or a dual problem. However, there has been no systematic comparison of these algorithms in terms of their speed, generalization performance (accuracy/likelihood) and their ability to achieve steady state generalization performance fast. With this aim, we compare different algorithms and make recommendations, useful for a practitioner. We conclude that the selection of an algorithm for sequence labeling depends on the evaluation criterion used and its implementation simplicity.