Transformer-based Model for ASR N-Best Rescoring and Rewriting

TL;DR

This paper introduces a Transformer-based model for on-device ASR N-best rescoring and rewriting, improving accuracy by leveraging full hypothesis context and a new training objective, leading to significant WER reduction.

Contribution

It presents a novel Transformer model with a discriminative training objective for combined rescoring and rewriting in on-device ASR systems, enhancing performance.

Findings

Achieves up to 8.6% relative WER reduction.

Outperforms rescoring-only baseline.

Effective for both rescoring and rewriting tasks.

Abstract

Voice assistants increasingly use on-device Automatic Speech Recognition (ASR) to ensure speed and privacy. However, due to resource constraints on the device, queries pertaining to complex information domains often require further processing by a search engine. For such applications, we propose a novel Transformer based model capable of rescoring and rewriting, by exploring full context of the N-best hypotheses in parallel. We also propose a new discriminative sequence training objective that can work well for both rescore and rewrite tasks. We show that our Rescore+Rewrite model outperforms the Rescore-only baseline, and achieves up to an average 8.6% relative Word Error Rate (WER) reduction over the ASR system by itself.