Streaming Punctuation for Long-form Dictation with Transformers

TL;DR

This paper introduces a streaming punctuation method for long-form speech dictation using transformers, improving segmentation and punctuation accuracy in real-time ASR systems, and enhancing downstream machine translation quality.

Contribution

It presents a novel streaming approach with dynamic decoding windows for punctuation in ASR, addressing real-time constraints and improving segmentation and translation performance.

Findings

Segmentation F0.5-score improved by 13.9%.

BLEU-score for MT increased by 0.66 on average.

Effective long-context punctuation in real-time ASR.

Abstract

While speech recognition Word Error Rate (WER) has reached human parity for English, long-form dictation scenarios still suffer from segmentation and punctuation problems resulting from irregular pausing patterns or slow speakers. Transformer sequence tagging models are effective at capturing long bi-directional context, which is crucial for automatic punctuation. Automatic Speech Recognition (ASR) production systems, however, are constrained by real-time requirements, making it hard to incorporate the right context when making punctuation decisions. In this paper, we propose a streaming approach for punctuation or re-punctuation of ASR output using dynamic decoding windows and measure its impact on punctuation and segmentation accuracy across scenarios. The new system tackles over-segmentation issues, improving segmentation F0.5-score by 13.9%. Streaming punctuation achieves an average BLEU-score improvement of 0.66 for the downstream task of Machine Translation (MT).