Local Equivariance Error-Based Metrics for Evaluating Sampling-Frequency-Independent Property of Neural Network

TL;DR

This paper introduces three new metrics based on local equivariance error to evaluate how well neural networks for audio processing maintain their performance across different sampling frequencies, addressing a gap in robustness assessment.

Contribution

The paper proposes novel metrics using local equivariance error to quantify sampling-frequency-independent robustness of neural networks in audio source separation.

Findings

Metrics strongly correlate with performance degradation at untrained sampling frequencies.

Proposed metrics effectively evaluate robustness of network components predicting time-frequency masks.

Extension of local equivariance error to measure signal resampling robustness in audio neural networks.

Abstract

Audio signal processing methods based on deep neural networks (DNNs) are typically trained only at a single sampling frequency (SF) and therefore require signal resampling to handle untrained SFs. However, recent studies have shown that signal resampling can degrade performance with untrained SFs. This problem has been overlooked because most studies evaluate only the performance at trained SFs. In this paper, to assess the robustness of DNNs to SF changes, which we refer to as the SF-independent (SFI) property, we propose three metrics to quantify the SFI property on the basis of local equivariance error (LEE). LEE measures the robustness of DNNs to input transformations. By using signal resampling as input transformation, we extend LEE to measure the robustness of audio source separation methods to signal resampling. The proposed metrics are constructed to quantify the SFI property in specific network components responsible for predicting time-frequency masks. Experiments on music source separation demonstrated a strong correlation between the proposed metrics and performance degradation at untrained SFs.