Imputer: Sequence Modelling via Imputation and Dynamic Programming

TL;DR

The paper introduces the Imputer, a neural sequence model that generates sequences through iterative imputations, enabling efficient training and competitive performance in speech recognition tasks.

Contribution

It proposes a novel iterative generative model with a dynamic programming training algorithm that marginalizes over alignments and generation orders.

Findings

Outperforms prior non-autoregressive models in speech recognition

Achieves 11.1 WER on LibriSpeech test-other, better than CTC and seq2seq

Requires only a constant number of generation steps regardless of sequence length

Abstract

This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations. The Imputer is an iterative generative model, requiring only a constant number of generation steps independent of the number of input or output tokens. The Imputer can be trained to approximately marginalize over all possible alignments between the input and output sequences, and all possible generation orders. We present a tractable dynamic programming training algorithm, which yields a lower bound on the log marginal likelihood. When applied to end-to-end speech recognition, the Imputer outperforms prior non-autoregressive models and achieves competitive results to autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1 WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER.