Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation

TL;DR

This paper introduces a novel mask-weighted spatial likelihood coding method that jointly estimates speech source locations and masks, improving performance in challenging acoustic environments and replacing traditional localization systems.

Contribution

The paper proposes a new encoding technique for joint localization and mask estimation, demonstrating superior results and a universal approach adaptable to various scenarios.

Findings

Significant performance improvement over baseline methods

Effective joint estimation of localization and masks

Potential to replace traditional localization systems

Abstract

Due to their robustness and flexibility, neural-driven beamformers are a popular choice for speech separation in challenging environments with a varying amount of simultaneous speakers alongside noise and reverberation. Time-frequency masks and relative directions of the speakers regarding a fixed spatial grid can be used to estimate the beamformer's parameters. To some degree, speaker-independence is achieved by ensuring a greater amount of spatial partitions than speech sources. In this work, we analyze how to encode both mask and positioning into such a grid to enable joint estimation of both quantities. We propose mask-weighted spatial likelihood coding and show that it achieves considerable performance in both tasks compared to baseline encodings optimized for either localization or mask estimation. In the same setup, we demonstrate superiority for joint estimation of both quantities. Conclusively, we propose a universal approach which can replace an upstream sound source localization system solely by adapting the training framework, making it highly relevant in performance-critical scenarios.