A Parameter-efficient Language Extension Framework for Multilingual ASR

TL;DR

This paper introduces PELE, a parameter-efficient framework for extending multilingual speech recognition models to new languages, effectively addressing catastrophic forgetting and demonstrating superior performance with minimal additional parameters.

Contribution

The paper proposes a novel architecture-based framework, PELE, that enables efficient language extension in MASR models using parameter-efficient fine-tuning modules, outperforming traditional methods.

Findings

PELE effectively incorporates new languages with minimal performance loss.

Adapter-based PEFT modules outperform weight or input feature-based methods.

Experiments on 5 languages show superior results over joint learning approaches.

Abstract

Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning problem is probabilistically decomposed into language identity prediction (LP) and cross-lingual adaptation (XLA) sub-problems. Based on this, we propose an architecture-based framework for language extension that can fundamentally solve catastrophic forgetting, debudded as PELE. PELE is designed to be parameter-efficient, incrementally incorporating an add-on module to adapt to a new language. Specifically, different parameter-efficient fine-tuning (PEFT) modules and their variants are explored as potential candidates to perform XLA. Experiments are carried out on 5 new languages with a wide range of low-resourced data sizes. The best-performing PEFT candidate can achieve satisfactory performance across all languages and demonstrates superiority in three of five languages over the continual joint learning setting. Notably, PEFT methods focusing on weight parameters or input features are revealed to be limited in performance, showing significantly inferior extension capabilities compared to inserting a lightweight module in between layers such as an Adapter.