Active Learning of Non-semantic Speech Tasks with Pretrained Models

TL;DR

This paper introduces ALOE, an active learning system that leverages pretrained models to efficiently learn non-semantic speech tasks with limited labeled data, reducing the need for extensive annotation.

Contribution

ALOE is a novel active learning framework that improves data and label efficiency for non-semantic speech tasks using pretrained models and incremental data labeling.

Findings

ALOE achieves comparable performance to full-data baselines with less labeled data.

Using pretrained models with ALOE enhances data efficiency across various tasks.

ALOE's effectiveness is demonstrated with multiple acquisition functions and model architectures.

Abstract

Pretraining neural networks with massive unlabeled datasets has become popular as it equips the deep models with a better prior to solve downstream tasks. However, this approach generally assumes that the downstream tasks have access to annotated data of sufficient size. In this work, we propose ALOE, a novel system for improving the data- and label-efficiency of non-semantic speech tasks with active learning. ALOE uses pretrained models in conjunction with active learning to label data incrementally and learn classifiers for downstream tasks, thereby mitigating the need to acquire labeled data beforehand. We demonstrate the effectiveness of ALOE on a wide range of tasks, uncertainty-based acquisition functions, and model architectures. Training a linear classifier on top of a frozen encoder with ALOE is shown to achieve performance similar to several baselines that utilize the entire labeled data.