Efficient Sequence Labeling with Actor-Critic Training

TL;DR

This paper introduces an actor-critic training method for RNNs in sequence labeling, making them competitive with CRFs and outperforming them in some tasks by addressing exposure bias.

Contribution

It presents a novel actor-critic training approach for RNNs that improves sequence labeling performance and reduces exposure bias compared to existing methods.

Findings

AC-RNN matches CRF performance on NER and CCG tagging

AC-RNN outperforms CRF on Machine Transliteration

Training strategy surpasses Scheduled Sampling and Self-Critical methods

Abstract

Neural approaches to sequence labeling often use a Conditional Random Field (CRF) to model their output dependencies, while Recurrent Neural Networks (RNN) are used for the same purpose in other tasks. We set out to establish RNNs as an attractive alternative to CRFs for sequence labeling. To do so, we address one of the RNN's most prominent shortcomings, the fact that it is not exposed to its own errors with the maximum-likelihood training. We frame the prediction of the output sequence as a sequential decision-making process, where we train the network with an adjusted actor-critic algorithm (AC-RNN). We comprehensively compare this strategy with maximum-likelihood training for both RNNs and CRFs on three structured-output tasks. The proposed AC-RNN efficiently matches the performance of the CRF on NER and CCG tagging, and outperforms it on Machine Transliteration. We also show that our training strategy is significantly better than other techniques for addressing RNN's exposure bias, such as Scheduled Sampling, and Self-Critical policy training.