In-context Language Learning for Endangered Languages in Speech Recognition

TL;DR

This paper explores how large language models can learn to perform speech recognition on endangered, low-resource languages through in-context learning, improving performance without supervised training.

Contribution

It demonstrates that in-context learning enables LLMs to recognize unseen languages in speech tasks, outperforming traditional instruction-based methods and rivaling dedicated models.

Findings

More relevant text samples improve recognition accuracy.

Probability-based ICL outperforms instruction-based ICL.

LLMs can achieve comparable or better ASR performance on endangered languages.

Abstract

With approximately 7,000 languages spoken worldwide, current large language models (LLMs) support only a small subset. Prior research indicates LLMs can learn new languages for certain tasks without supervised data. We extend this investigation to speech recognition, investigating whether LLMs can learn unseen, low-resource languages through in-context learning (ICL). With experiments on four diverse endangered languages that LLMs have not been trained on, we find that providing more relevant text samples enhances performance in both language modelling and Automatic Speech Recognition (ASR) tasks. Furthermore, we show that the probability-based approach outperforms the traditional instruction-based approach in language learning. Lastly, we show ICL enables LLMs to achieve ASR performance that is comparable to or even surpasses dedicated language models trained specifically for these languages, while preserving the original capabilities of the LLMs. Our code is publicly available.