A Brief Overview of Unsupervised Neural Speech Representation Learning

TL;DR

This paper reviews the development of unsupervised neural speech representation learning, focusing on self-supervised and probabilistic models, highlighting their differences, challenges, and progress over the past decade.

Contribution

It provides a comprehensive taxonomy and comparison of unsupervised speech representation models, summarizing recent advancements and unique challenges in the field.

Findings

Self-supervised and probabilistic models are the main categories.

Recent progress has improved speech representation quality.

Distinct challenges remain for speech data compared to other domains.

Abstract

Unsupervised representation learning for speech processing has matured greatly in the last few years. Work in computer vision and natural language processing has paved the way, but speech data offers unique challenges. As a result, methods from other domains rarely translate directly. We review the development of unsupervised representation learning for speech over the last decade. We identify two primary model categories: self-supervised methods and probabilistic latent variable models. We describe the models and develop a comprehensive taxonomy. Finally, we discuss and compare models from the two categories.