Dissecting Performance Degradation in Audio Source Separation under Sampling Frequency Mismatch

TL;DR

This paper investigates why audio source separation performance degrades when using sampling frequency mismatch and proposes novel resampling methods that improve robustness across models.

Contribution

It introduces noisy-kernel and trainable-kernel resampling techniques that mitigate performance loss due to sampling frequency mismatch in neural network-based audio processing.

Findings

Noisy-kernel resampling improves separation quality.

Trainable-kernel adapts to different models effectively.

Proposed methods outperform conventional resampling.

Abstract

Audio processing methods based on deep neural networks are typically trained at a single sampling frequency (SF). To handle untrained SFs, signal resampling is commonly employed, but it can degrade performance, particularly when the input SF is lower than the trained SF. This paper investigates the causes of this degradation through two hypotheses: (i) the lack of high-frequency components introduced by up-sampling, and (ii) the greater importance of their presence than their precise representation. To examine these hypotheses, we compare conventional resampling with three alternatives: post-resampling noise addition, which adds Gaussian noise to the resampled signal; noisy-kernel resampling, which perturbs the kernel with Gaussian noise to enrich high-frequency components; and trainable-kernel resampling, which adapts the interpolation kernel through training. Experiments on music source separation show that noisy-kernel and trainable-kernel resampling alleviate the degradation observed with conventional resampling. We further demonstrate that noisy-kernel resampling is effective across diverse models, highlighting it as a simple yet practical option.