Where's That Voice Coming? Continual Learning for Sound Source Localization

TL;DR

This paper introduces CL-SSL, a continual learning approach for sound source localization that adapts to changing acoustic environments without catastrophic forgetting, maintaining high accuracy with minimal parameter growth.

Contribution

It proposes an exemplar-free continual learning method with task-specific sub-networks and a scaling mechanism to improve SSL in dynamic conditions.

Findings

Maintains high localization accuracy across diverse environments

Uses minimal additional parameters for incremental learning

Effective on both simulated and real-world data

Abstract

Sound source localization (SSL) is essential for many speech-processing applications. Deep learning models have achieved high performance, but often fail when the training and inference environments differ. Adapting SSL models to dynamic acoustic conditions faces a major challenge: catastrophic forgetting. In this work, we propose an exemplar-free continual learning strategy for SSL (CL-SSL) to address such a forgetting phenomenon. CL-SSL applies task-specific sub-networks to adapt across diverse acoustic environments while retaining previously learned knowledge. It also uses a scaling mechanism to limit parameter growth, ensuring consistent performance across incremental tasks. We evaluated CL-SSL on simulated data with varying microphone distances and real-world data with different noise levels. The results demonstrate CL-SSL's ability to maintain high accuracy with minimal parameter increase, offering an efficient solution for SSL applications.