Segmenting Subtitles for Correcting ASR Segmentation Errors

TL;DR

This paper introduces a neural tagging model that uses subtitles as proxy data to correct ASR segmentation errors, enhancing downstream translation and retrieval tasks in low-resource languages.

Contribution

It presents a novel approach leveraging subtitles and synthetic data to improve ASR segmentation correction for low-resource languages.

Findings

Improved MT performance after segmentation correction

Enhanced cross-language information retrieval accuracy

Effective use of subtitles as proxy data for training

Abstract

Typical ASR systems segment the input audio into utterances using purely acoustic information, which may not resemble the sentence-like units that are expected by conventional machine translation (MT) systems for Spoken Language Translation. In this work, we propose a model for correcting the acoustic segmentation of ASR models for low-resource languages to improve performance on downstream tasks. We propose the use of subtitles as a proxy dataset for correcting ASR acoustic segmentation, creating synthetic acoustic utterances by modeling common error modes. We train a neural tagging model for correcting ASR acoustic segmentation and show that it improves downstream performance on MT and audio-document cross-language information retrieval (CLIR).