Text Editing as Imitation Game

TL;DR

This paper introduces a novel approach to text editing by framing it as an imitation game with behavioral cloning, enabling more flexible and efficient sequence-level action generation, and demonstrates superior performance on arithmetic benchmarks.

Contribution

It reformulates text editing as an imitation game with a dual decoder structure and trajectory augmentation, improving flexibility and robustness over traditional token-level tagging methods.

Findings

Outperforms autoregressive baselines on arithmetic benchmarks

Enhances efficiency and robustness of text editing models

Introduces a flexible, sequence-level action generation framework

Abstract

Text editing, such as grammatical error correction, arises naturally from imperfect textual data. Recent works frame text editing as a multi-round sequence tagging task, where operations -- such as insertion and substitution -- are represented as a sequence of tags. While achieving good results, this encoding is limited in flexibility as all actions are bound to token-level tags. In this work, we reformulate text editing as an imitation game using behavioral cloning. Specifically, we convert conventional sequence-to-sequence data into state-to-action demonstrations, where the action space can be as flexible as needed. Instead of generating the actions one at a time, we introduce a dual decoders structure to parallel the decoding while retaining the dependencies between action tokens, coupled with trajectory augmentation to alleviate the distribution shift that imitation learning often suffers. In experiments on a suite of Arithmetic Equation benchmarks, our model consistently outperforms the autoregressive baselines in terms of performance, efficiency, and robustness. We hope our findings will shed light on future studies in reinforcement learning applying sequence-level action generation to natural language processing.