An Adapter-Based Unified Model for Multiple Spoken Language Processing Tasks

TL;DR

This paper proposes an adapter-based fine-tuning approach to develop a unified speech processing model that efficiently handles multiple tasks, achieving significant performance improvements on the SUPERB benchmark.

Contribution

It introduces a novel adapter-based fine-tuning method for creating a single model capable of multiple spoken language tasks, reducing computational resources.

Findings

Achieves an average of 18.4% improvement across five speech tasks.

Enables a single encoder-decoder model to perform multiple tasks effectively.

Maintains efficiency in parameter updates during training.

Abstract

Self-supervised learning models have revolutionized the field of speech processing. However, the process of fine-tuning these models on downstream tasks requires substantial computational resources, particularly when dealing with multiple speech-processing tasks. In this paper, we explore the potential of adapter-based fine-tuning in developing a unified model capable of effectively handling multiple spoken language processing tasks. The tasks we investigate are Automatic Speech Recognition, Phoneme Recognition, Intent Classification, Slot Filling, and Spoken Emotion Recognition. We validate our approach through a series of experiments on the SUPERB benchmark, and our results indicate that adapter-based fine-tuning enables a single encoder-decoder model to perform multiple speech processing tasks with an average improvement of 18.4% across the five target tasks while staying efficient in terms of parameter updates.