Multitask Learning for Low Resource Spoken Language Understanding

TL;DR

This paper investigates how multitask learning enhances low-resource spoken language understanding by training models on speech recognition and classification tasks, showing significant improvements especially with minimal data.

Contribution

It demonstrates the effectiveness of multitask learning in low-resource scenarios and compares different task configurations for speech and text classification.

Findings

Multitask models outperform end-to-end intent classification models.

Models trained with few examples per class perform competitively with text-based baselines.

Multitask learning matches large models' performance with fewer parameters.

Abstract

We explore the benefits that multitask learning offer to speech processing as we train models on dual objectives with automatic speech recognition and intent classification or sentiment classification. Our models, although being of modest size, show improvements over models trained end-to-end on intent classification. We compare different settings to find the optimal disposition of each task module compared to one another. Finally, we study the performance of the models in low-resource scenario by training the models with as few as one example per class. We show that multitask learning in these scenarios compete with a baseline model trained on text features and performs considerably better than a pipeline model. On sentiment classification, we match the performance of an end-to-end model with ten times as many parameters. We consider 4 tasks and 4 datasets in Dutch and English.