End-to-end Two-dimensional Sound Source Localization With Ad-hoc Microphone Arrays

TL;DR

This paper introduces a deep learning approach for 2D sound source localization using ad-hoc microphone arrays, capable of working with randomly distributed microphones in indoor environments.

Contribution

It presents an end-to-end spatial-temporal deep model with attention architecture that handles variable microphone configurations and localizes speakers in complex acoustic conditions.

Findings

High accuracy in reverberant environments

Effective with a single microphone per node

Handles variable microphone array configurations

Abstract

Conventional sound source localization methods are mostly based on a single microphone array that consists of multiple microphones. They are usually formulated as the estimation of the direction of arrival problem. In this paper, we propose a deep-learning-based end-to-end sound source localization method with ad-hoc microphone arrays, where an ad-hoc microphone array is a set of randomly distributed microphone arrays that collaborate with each other. It can produce two-dimensional locations of speakers with only a single microphone per node. Specifically, we divide a targeted indoor space into multiple local areas. We encode each local area by a one-hot code, therefore, the node and speaker locations can be represented by the one-hot codes. Accordingly, the sound source localization problem is formulated as such a classification task of recognizing the one-hot code of the speaker given the one hot codes of the microphone nodes and their speech recordings. An end-to-end spatial-temporal deep model is designed for the classification problem. It utilizes a spatial-temporal attention architecture with a fusion layer inserted in the middle of the architecture, which is able to handle arbitrarily different numbers of microphone nodes during the model training and test. Experimental results show that the proposed method yields good performance in highly reverberant and noisy environments.