Improving Punctuation Restoration for Speech Transcripts via External Data

TL;DR

This paper improves punctuation restoration in speech transcripts, especially noisy ones, by using external data sampling and a two-stage fine-tuning approach with BERT, leading to better performance.

Contribution

It introduces a novel data sampling technique and a two-stage fine-tuning method to enhance punctuation restoration for noisy speech transcripts.

Findings

Achieved 1.12% higher F1 score over baseline.

Effective use of external data improves punctuation accuracy.

Method is particularly beneficial for noisy speech scenarios.

Abstract

Automatic Speech Recognition (ASR) systems generally do not produce punctuated transcripts. To make transcripts more readable and follow the expected input format for downstream language models, it is necessary to add punctuation marks. In this paper, we tackle the punctuation restoration problem specifically for the noisy text (e.g., phone conversation scenarios). To leverage the available written text datasets, we introduce a data sampling technique based on an n-gram language model to sample more training data that are similar to our in-domain data. Moreover, we propose a two-stage fine-tuning approach that utilizes the sampled external data as well as our in-domain dataset for models based on BERT. Extensive experiments show that the proposed approach outperforms the baseline with an improvement of 1:12% F1 score.