A Parallelizable Lattice Rescoring Strategy with Neural Language Models

TL;DR

This paper introduces a parallelizable lattice rescoring method using neural language models that improves efficiency and lattice compactness in speech recognition, facilitating easier integration with existing systems.

Contribution

It presents a novel parallel computation strategy and lattice expansion algorithm that enhance neural LM rescoring efficiency and lattice compactness.

Findings

Achieves comparable recognition performance to baseline methods.

Generates more compact lattices, reducing computational complexity.

Simplifies integration of neural LMs with Kaldi.

Abstract

This paper proposes a parallel computation strategy and a posterior-based lattice expansion algorithm for efficient lattice rescoring with neural language models (LMs) for automatic speech recognition. First, lattices from first-pass decoding are expanded by the proposed posterior-based lattice expansion algorithm. Second, each expanded lattice is converted into a minimal list of hypotheses that covers every arc. Each hypothesis is constrained to be the best path for at least one arc it includes. For each lattice, the neural LM scores of the minimal list are computed in parallel and are then integrated back to the lattice in the rescoring stage. Experiments on the Switchboard dataset show that the proposed rescoring strategy obtains comparable recognition performance and generates more compact lattices than a competitive baseline method. Furthermore, the parallel rescoring method offers more flexibility by simplifying the integration of PyTorch-trained neural LMs for lattice rescoring with Kaldi.