Improving Autoregressive Training with Dynamic Oracles

TL;DR

This paper introduces novel dynamic oracles for autoregressive training that address exposure bias and metric mismatch issues, improving performance in NER and text summarization tasks by maintaining no-regret guarantees for decomposable metrics.

Contribution

It develops new dynamic oracles for metrics like span-based F1, ROUGE, and BLEU, extending DAgger's applicability to a broader range of NLP tasks.

Findings

Outperforms baseline methods in NER and summarization

Less effective in machine translation

Maintains no-regret guarantees for decomposable metrics

Abstract

Many tasks within NLP can be framed as sequential decision problems, ranging from sequence tagging to text generation. However, for many tasks, the standard training methods, including maximum likelihood (teacher forcing) and scheduled sampling, suffer from exposure bias and a mismatch between metrics employed during training and inference. DAgger provides a solution to mitigate these problems, yet it requires a metric-specific dynamic oracle algorithm, which does not exist for many common metrics like span-based F1, ROUGE, and BLEU. In this paper, we develop these novel dynamic oracles and show they maintain DAgger's no-regret guarantee for decomposable metrics like span-based F1. We evaluate the algorithm's performance on named entity recognition (NER), text summarization, and machine translation (MT). While DAgger with dynamic oracle yields less favorable results in our MT experiments, it outperforms the baseline techniques in NER and text summarization.