Boosting Punctuation Restoration with Data Generation and Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach combined with large generative language models to improve punctuation restoration in ASR texts, addressing the data discrepancy issue and achieving state-of-the-art results.

Contribution

It presents a novel reinforcement learning method leveraging pre-trained generative models to enhance punctuation restoration for ASR outputs.

Findings

Achieved state-of-the-art performance on two benchmark datasets.

Effectively bridged the gap between written and ASR texts.

Improved punctuation accuracy in ASR transcripts.

Abstract

Punctuation restoration is an important task in automatic speech recognition (ASR) which aim to restore the syntactic structure of generated ASR texts to improve readability. While punctuated texts are abundant from written documents, the discrepancy between written punctuated texts and ASR texts limits the usability of written texts in training punctuation restoration systems for ASR texts. This paper proposes a reinforcement learning method to exploit in-topic written texts and recent advances in large pre-trained generative language models to bridge this gap. The experiments show that our method achieves state-of-the-art performance on the ASR test set on two benchmark datasets for punctuation restoration.