Joint prediction of truecasing and punctuation for conversational speech in low-resource scenarios

TL;DR

This paper introduces a multi-task model for joint truecasing and punctuation prediction in conversational speech, leveraging transfer learning from written text to improve performance in low-resource scenarios.

Contribution

It proposes a novel multi-task approach that exploits the relationship between casing and punctuation, and demonstrates effective transfer learning from written to conversational text domains.

Findings

Transfer learning reduces data requirements for conversational models.

Cross-domain training improves punctuation and casing prediction accuracy.

Multi-task learning exploits the correlation between casing and punctuation.

Abstract

Capitalization and punctuation are important cues for comprehending written texts and conversational transcripts. Yet, many ASR systems do not produce punctuated and case-formatted speech transcripts. We propose to use a multi-task system that can exploit the relations between casing and punctuation to improve their prediction performance. Whereas text data for predicting punctuation and truecasing is seemingly abundant, we argue that written text resources are inadequate as training data for conversational models. We quantify the mismatch between written and conversational text domains by comparing the joint distributions of punctuation and word cases, and by testing our model cross-domain. Further, we show that by training the model in the written text domain and then transfer learning to conversations, we can achieve reasonable performance with less data.