Sequence Transduction with Graph-based Supervision

TL;DR

This paper introduces a generalized transducer loss function that uses graph-based supervision, leading to improved speech recognition accuracy and more flexible alignment control compared to traditional RNN-T models.

Contribution

It proposes a new transducer objective that accepts graph representations of labels, enabling flexible manipulation of training lattices and better optimization.

Findings

Achieves 4.8% relative improvement on LibriSpeech test-other

Ensures strictly monotonic alignments for better decoding

Demonstrates the effectiveness of graph-based supervision in transducer models

Abstract

The recurrent neural network transducer (RNN-T) objective plays a major role in building today's best automatic speech recognition (ASR) systems for production. Similarly to the connectionist temporal classification (CTC) objective, the RNN-T loss uses specific rules that define how a set of alignments is generated to form a lattice for the full-sum training. However, it is yet largely unknown if these rules are optimal and do lead to the best possible ASR results. In this work, we present a new transducer objective function that generalizes the RNN-T loss to accept a graph representation of the labels, thus providing a flexible and efficient framework to manipulate training lattices, e.g., for studying different transition rules, implementing different transducer losses, or restricting alignments. We demonstrate that transducer-based ASR with CTC-like lattice achieves better results compared to standard RNN-T, while also ensuring a strictly monotonic alignment, which will allow better optimization of the decoding procedure. For example, the proposed CTC-like transducer achieves an improvement of 4.8% on the test-other condition of LibriSpeech relative to an equivalent RNN-T based system.