Upsampling layers for music source separation

TL;DR

This paper investigates the impact of various upsampling layers and artifacts on music source separation, benchmarking multiple methods including novel layers, and finds that perceptually preferable filtering artifacts often lead to worse objective scores.

Contribution

It introduces and benchmarks new upsampling layers, including novel stretch, sinc, and learnable wavelet methods, analyzing their artifacts and effects on audio quality and model performance.

Findings

Filtering artifacts are perceptually preferable despite lower objective scores.

Different upsampling methods produce distinct spectral and tonal artifacts.

Novel wavelet-based upsampling layers are proposed and evaluated.

Abstract

Upsampling artifacts are caused by problematic upsampling layers and due to spectral replicas that emerge while upsampling. Also, depending on the used upsampling layer, such artifacts can either be tonal artifacts (additive high-frequency noise) or filtering artifacts (substractive, attenuating some bands). In this work we investigate the practical implications of having upsampling artifacts in the resulting audio, by studying how different artifacts interact and assessing their impact on the models' performance. To that end, we benchmark a large set of upsampling layers for music source separation: different transposed and subpixel convolution setups, different interpolation upsamplers (including two novel layers based on stretch and sinc interpolation), and different wavelet-based upsamplers (including a novel learnable wavelet layer). Our results show that filtering artifacts, associated with interpolation upsamplers, are perceptually preferrable, even if they tend to achieve worse objective scores.