XANE: eXplainable Acoustic Neural Embeddings

TL;DR

This paper introduces XANE, a method for extracting explainable neural embeddings that model background acoustics, enabling high-accuracy acoustic parameter estimation and outperforming existing embeddings in clustering tasks.

Contribution

XANE provides a novel approach to generate explainable acoustic embeddings that estimate background acoustic parameters non-intrusively, improving clustering performance and interpretability.

Findings

Achieves a mean F1 score of 95.2% on three tasks.

Accurately estimates 14 acoustic parameters.

Operates with a real-time factor 17 times lower than baselines.

Abstract

We present a novel method for extracting neural embeddings that model the background acoustics of a speech signal. The extracted embeddings are used to estimate specific parameters related to the background acoustic properties of the signal in a non-intrusive manner, which allows the embeddings to be explainable in terms of those parameters. We illustrate the value of these embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 95.2\% for three different tasks, outperforming significantly the WavLM based signal embeddings. We also show that the proposed method can explain the embeddings by estimating 14 acoustic parameters characterizing the background acoustics, including reverberation and noise levels, overlapped speech detection, CODEC type detection and noise type detection with high accuracy and a real-time factor 17 times lower than an external baseline method.