Continual Adaptation for Pacific Indigenous Speech Recognition

TL;DR

This paper empirically investigates methods for adapting speech recognition models to low-resource Pacific Indigenous languages, highlighting challenges like data scarcity, catastrophic forgetting, and the need for robust continual learning strategies.

Contribution

It provides a comprehensive analysis of adaptation strategies like Full Fine-Tuning and LoRA, and explores continual learning frameworks for multiple languages in low-resource settings.

Findings

LoRA adapts well initially but suffers from catastrophic forgetting

Adapting to distant languages causes internal representational drift

Robust adaptation strategies are urgently needed for underrepresented languages

Abstract

Speech foundation models struggle with low-resource Pacific Indigenous languages because of severe data scarcity. Furthermore, full fine-tuning risks catastrophic forgetting. To address this gap, we present an empirical study adapting models to real-world Pacific datasets. We investigate how data volume and linguistic features affect adaptation success. Specifically, we evaluate strategies including Full Fine-Tuning and Low-Rank Adaptation (LoRA). Additionally, we analyze a continual learning framework for sequentially acquiring multiple languages. We demonstrate that adapting to these distant languages causes severe internal representational drift. Consequently, these models face a strict plasticity and stability dilemma. While LoRA adapts well initially, it suffers from catastrophic forgetting during sequential learning. Ultimately, this study highlights the urgent need for robust adaptation strategies tailored to underrepresented languages.