Audio Source Separation with Discriminative Scattering Networks

TL;DR

This paper introduces a multi-resolution wavelet scattering representation for single-channel audio source separation, demonstrating improved results over fixed-resolution methods and exploring discriminative training with neural networks.

Contribution

It proposes a novel multi-resolution wavelet scattering approach for audio separation and integrates it into discriminative neural network training regimes, advancing the state of the art.

Findings

Multi-resolution scattering improves source separation performance.

Discriminative training with neural networks enhances separation quality.

The approach generalizes Constant Q Transforms with additional convolution layers.

Abstract

In this report we describe an ongoing line of research for solving single-channel source separation problems. Many monaural signal decomposition techniques proposed in the literature operate on a feature space consisting of a time-frequency representation of the input data. A challenge faced by these approaches is to effectively exploit the temporal dependencies of the signals at scales larger than the duration of a time-frame. In this work we propose to tackle this problem by modeling the signals using a time-frequency representation with multiple temporal resolutions. The proposed representation consists of a pyramid of wavelet scattering operators, which generalizes Constant Q Transforms (CQT) with extra layers of convolution and complex modulus. We first show that learning standard models with this multi-resolution setting improves source separation results over fixed-resolution methods. As study case, we use Non-Negative Matrix Factorizations (NMF) that has been widely considered in many audio application. Then, we investigate the inclusion of the proposed multi-resolution setting into a discriminative training regime. We discuss several alternatives using different deep neural network architectures.