PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement

TL;DR

This paper introduces PAAPLoss, a novel auxiliary loss function for speech enhancement that aligns acoustic parameters with perceptual quality, improving output naturalness and interpretability.

Contribution

The paper proposes a neural network-based method to estimate non-differentiable acoustic parameters and incorporates phoneme-specific weighting to enhance speech quality.

Findings

Improves speech enhancement performance in both time and frequency domains

Enhances perceptual quality as measured by standard metrics

Provides interpretability through phoneme-dependent analysis

Abstract

Despite rapid advancement in recent years, current speech enhancement models often produce speech that differs in perceptual quality from real clean speech. We propose a learning objective that formalizes differences in perceptual quality, by using domain knowledge of acoustic-phonetics. We identify temporal acoustic parameters -- such as spectral tilt, spectral flux, shimmer, etc. -- that are non-differentiable, and we develop a neural network estimator that can accurately predict their time-series values across an utterance. We also model phoneme-specific weights for each feature, as the acoustic parameters are known to show different behavior in different phonemes. We can add this criterion as an auxiliary loss to any model that produces speech, to optimize speech outputs to match the values of clean speech in these features. Experimentally we show that it improves speech enhancement workflows in both time-domain and time-frequency domain, as measured by standard evaluation metrics. We also provide an analysis of phoneme-dependent improvement on acoustic parameters, demonstrating the additional interpretability that our method provides. This analysis can suggest which features are currently the bottleneck for improvement.