Learning and Analyzing Generation Order for Undirected Sequence Models

TL;DR

This paper introduces a reinforcement learning approach to determine optimal generation order in undirected neural sequence models, improving translation quality and revealing insights into their generation mechanisms.

Contribution

It presents a novel policy learning method for generation order in undirected models, outperforming heuristic orders and analyzing learned patterns.

Findings

Learned orders yield higher BLEU scores than traditional left-to-right decoding.

The policy often predicts outer-to-inner order, starting from the edges.

Predicted positions tend to form contiguous syntactic structures.

Abstract

Undirected neural sequence models have achieved performance competitive with the state-of-the-art directed sequence models that generate monotonically from left to right in machine translation tasks. In this work, we train a policy that learns the generation order for a pre-trained, undirected translation model via reinforcement learning. We show that the translations decoded by our learned orders achieve higher BLEU scores than the outputs decoded from left to right or decoded by the learned order from Mansimov et al. (2019) on the WMT'14 German-English translation task. On examples with a maximum source and target length of 30 from De-En, WMT'16 English-Romanian, and WMT'21 English-Chinese translation tasks, our learned order outperforms all heuristic generation orders on four out of six tasks. We next carefully analyze the learned order patterns via qualitative and quantitative analysis. We show that our policy generally follows an outer-to-inner order, predicting the left-most and right-most positions first, and then moving toward the middle while skipping less important words at the beginning. Furthermore, the policy usually predicts positions for a single syntactic constituent structure in consecutive steps. We believe our findings could provide more insights on the mechanism of undirected generation models and encourage further research in this direction. Our code is publicly available at https://github.com/jiangycTarheel/undirected-generation