Recent Advances in Direct Speech-to-text Translation

TL;DR

This survey reviews recent progress in direct speech-to-text translation, focusing on modeling, data, and application challenges, and discusses future research directions in the field.

Contribution

It provides a comprehensive categorization and analysis of current techniques addressing key challenges in direct speech translation, highlighting recent advances and future prospects.

Findings

Encoder-decoder and multitask frameworks address modeling burden.

Data augmentation, pre-training, and multilingual modeling mitigate data scarcity.

Application issues like real-time processing and gender bias are actively studied.

Abstract

Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly. In this paper, we present a comprehensive survey on direct speech translation aiming to summarize the current state-of-the-art techniques. First, we categorize the existing research work into three directions based on the main challenges -- modeling burden, data scarcity, and application issues. To tackle the problem of modeling burden, two main structures have been proposed, encoder-decoder framework (Transformer and the variants) and multitask frameworks. For the challenge of data scarcity, recent work resorts to many sophisticated techniques, such as data augmentation, pre-training, knowledge distillation, and multilingual modeling. We analyze and summarize the application issues, which include real-time, segmentation, named entity, gender bias, and code-switching. Finally, we discuss some promising directions for future work.