Fixing exposure bias with imitation learning needs powerful oracles

TL;DR

This paper explores using imitation learning with error-correcting oracles to address exposure bias in neural machine translation, but finds that highly performant SMT-based oracles may be too specialized for effective IL training.

Contribution

It demonstrates the challenges of applying powerful SMT-based oracles in IL for NMT exposure bias correction due to their pruning and idiosyncrasies.

Findings

SMT lattice-based oracle performs well in unconstrained translation

Pruned and idiosyncratic SMT oracles are ineffective for IL

Imitation learning requires more generalizable oracles for NMT

Abstract

We apply imitation learning (IL) to tackle the NMT exposure bias problem with error-correcting oracles, and evaluate an SMT lattice-based oracle which, despite its excellent performance in an unconstrained oracle translation task, turned out to be too pruned and idiosyncratic to serve as the oracle for IL.