Online Localization and Tracking of Multiple Moving Speakers in Reverberant Environments

TL;DR

This paper introduces an online method for localizing and tracking multiple moving speakers in reverberant environments using a robust acoustic feature, DP-RTF, combined with Bayesian and variational techniques for efficient source association and tracking.

Contribution

It proposes a novel online algorithm leveraging DP-RTF and a Bayesian variational framework for real-time multi-speaker localization and tracking in reverberant spaces.

Findings

Effective in real environments with reverberation

Accurate tracking of multiple moving speakers

Computationally efficient for real-time applications

Abstract

We address the problem of online localization and tracking of multiple moving speakers in reverberant environments. The paper has the following contributions. We use the direct-path relative transfer function (DP-RTF), an inter-channel feature that encodes acoustic information robust against reverberation, and we propose an online algorithm well suited for estimating DP-RTFs associated with moving audio sources. Another crucial ingredient of the proposed method is its ability to properly assign DP-RTFs to audio-source directions. Towards this goal, we adopt a maximum-likelihood formulation and we propose to use an exponentiated gradient (EG) to efficiently update source-direction estimates starting from their currently available values. The problem of multiple speaker tracking is computationally intractable because the number of possible associations between observed source directions and physical speakers grows exponentially with time. We adopt a Bayesian framework and we propose a variational approximation of the posterior filtering distribution associated with multiple speaker tracking, as well as an efficient variational expectation-maximization (VEM) solver. The proposed online localization and tracking method is thoroughly evaluated using two datasets that contain recordings performed in real environments.