Audio Self-supervised Learning: A Survey

TL;DR

This survey reviews the development, methods, benchmarks, and future challenges of self-supervised learning in audio and speech processing, highlighting its growing importance and current research landscape.

Contribution

It provides a comprehensive overview of audio SSL techniques, benchmarks, and open problems, filling a gap in current literature.

Findings

Summarizes various SSL methods for audio and speech

Identifies key benchmarks for evaluating audio SSL

Discusses open challenges and future directions

Abstract

Inspired by the humans' cognitive ability to generalise knowledge and skills, Self-Supervised Learning (SSL) targets at discovering general representations from large-scale data without requiring human annotations, which is an expensive and time consuming task. Its success in the fields of computer vision and natural language processing have prompted its recent adoption into the field of audio and speech processing. Comprehensive reviews summarising the knowledge in audio SSL are currently missing. To fill this gap, in the present work, we provide an overview of the SSL methods used for audio and speech processing applications. Herein, we also summarise the empirical works that exploit the audio modality in multi-modal SSL frameworks, and the existing suitable benchmarks to evaluate the power of SSL in the computer audition domain. Finally, we discuss some open problems and point out the future directions on the development of audio SSL.