Inference Strategies for Machine Translation with Conditional Masking

TL;DR

This paper explores various inference strategies for conditional masked language models in machine translation, proposing a thresholding heuristic that improves over traditional mask-predict methods.

Contribution

It introduces a new inference heuristic based on thresholding, supported by analysis and experiments demonstrating its effectiveness.

Findings

Thresholding strategy outperforms mask-predict in translation quality.

Analyzes behavior of inference heuristics on machine translation tasks.

Shows that factorization of conditional probabilities does not harm performance.

Abstract

Conditional masked language model (CMLM) training has proven successful for non-autoregressive and semi-autoregressive sequence generation tasks, such as machine translation. Given a trained CMLM, however, it is not clear what the best inference strategy is. We formulate masked inference as a factorization of conditional probabilities of partial sequences, show that this does not harm performance, and investigate a number of simple heuristics motivated by this perspective. We identify a thresholding strategy that has advantages over the standard "mask-predict" algorithm, and provide analyses of its behavior on machine translation tasks.