Optimizing Estonian TV Subtitles with Semi-supervised Learning and LLMs

TL;DR

This paper introduces a semi-supervised learning approach using fine-tuned Whisper models and LLM-based post-editing to generate high-quality Estonian TV subtitles, showing significant improvements in subtitle accuracy.

Contribution

It combines pseudo-labeling with LLM-based post-editing to enhance subtitle quality, a novel integration for Estonian language content.

Findings

Pseudo-labeling improves subtitle quality with unlabeled data.

LLM-based editing at test time enhances accuracy.

Training-time LLM editing does not provide additional benefits.

Abstract

This paper presents an approach for generating high-quality, same-language subtitles for Estonian TV content. We fine-tune the Whisper model on human-generated Estonian subtitles and enhance it with iterative pseudo-labeling and large language model (LLM) based post-editing. Our experiments demonstrate notable subtitle quality improvement through pseudo-labeling with an unlabeled dataset. We find that applying LLM-based editing at test time enhances subtitle accuracy, while its use during training does not yield further gains. This approach holds promise for creating subtitle quality close to human standard and could be extended to real-time applications.