SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization

TL;DR

This paper introduces SRP-DNN, a deep learning approach that learns direct-path phase differences for localizing multiple moving sound sources, improving accuracy in noisy and reverberant environments.

Contribution

The paper proposes a novel neural network architecture to directly learn and utilize direct-path phase differences for multi-source localization, addressing assignment ambiguity and source interaction issues.

Findings

Outperforms existing methods in noisy environments

Effective in real-world reverberant scenarios

Accurately localizes multiple moving sources

Abstract

Multiple moving sound source localization in real-world scenarios remains a challenging issue due to interaction between sources, time-varying trajectories, distorted spatial cues, etc. In this work, we propose to use deep learning techniques to learn competing and time-varying direct-path phase differences for localizing multiple moving sound sources. A causal convolutional recurrent neural network is designed to extract the direct-path phase difference sequence from signals of each microphone pair. To avoid the assignment ambiguity and the problem of uncertain output-dimension encountered when simultaneously predicting multiple targets, the learning target is designed in a weighted sum format, which encodes source activity in the weight and direct-path phase differences in the summed value. The learned direct-path phase differences for all microphone pairs can be directly used to construct the spatial spectrum according to the formulation of steered response power (SRP). This deep neural network (DNN) based SRP method is referred to as SRP-DNN. The locations of sources are estimated by iteratively detecting and removing the dominant source from the spatial spectrum, in which way the interaction between sources is reduced. Experimental results on both simulated and real-world data show the superiority of the proposed method in the presence of noise and reverberation.