Weakly Supervised Data Refinement and Flexible Sequence Compression for Efficient Thai LLM-based ASR

TL;DR

This paper introduces EThai-ASR, an efficient Thai speech recognition system leveraging large language models, with novel data refinement and sequence compression techniques to improve accuracy and reduce computational costs in low-resource settings.

Contribution

It is the first to apply LLMs to Thai ASR and proposes a weakly supervised data refinement and flexible sequence compression method for efficiency.

Findings

Achieved state-of-the-art accuracy on multiple datasets.

Reduced computational demands through sequence compression.

Enhanced speech encoder via self-evolving data refinement.

Abstract

Despite remarkable achievements, automatic speech recognition (ASR) in low-resource scenarios still faces two challenges: high-quality data scarcity and high computational demands. This paper proposes EThai-ASR, the first to apply large language models (LLMs) to Thai ASR and create an efficient LLM-based ASR system. EThai-ASR comprises a speech encoder, a connection module and a Thai LLM decoder. To address the data scarcity and obtain a powerful speech encoder, EThai-ASR introduces a self-evolving data refinement strategy to refine weak labels, yielding an enhanced speech encoder. Moreover, we propose a pluggable sequence compression module used in the connection module with three modes designed to reduce the sequence length, thus decreasing computational demands while maintaining decent performance. Extensive experiments demonstrate that EThai-ASR has achieved state-of-the-art accuracy in multiple datasets. We release our refined text transcripts to promote further research.