Semi-supervised source localization with deep generative modeling

TL;DR

This paper introduces a semi-supervised deep generative modeling approach using variational autoencoders for source localization in reverberant environments, effectively leveraging unlabeled data to improve accuracy.

Contribution

It presents a novel semi-supervised learning method with convolutional VAEs for source localization, outperforming traditional methods in label-scarce scenarios.

Findings

VAE-SSL outperforms SRP-PHAT and CNN in limited-label settings.

The approach effectively models phase of relative transfer functions.

Semi-supervised learning improves localization accuracy with less labeled data.

Abstract

We propose a semi-supervised localization approach based on deep generative modeling with variational autoencoders (VAEs). Localization in reverberant environments remains a challenge, which machine learning (ML) has shown promise in addressing. Even with large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. We address this issue by performing semi-supervised learning (SSL) with convolutional VAEs. The VAE is trained to generate the phase of relative transfer functions (RTFs), in parallel with a DOA classifier, on both labeled and unlabeled RTF samples. The VAE-SSL approach is compared with SRP-PHAT and fully-supervised CNNs. We find that VAE-SSL can outperform both SRP-PHAT and CNN in label-limited scenarios.