WildMix Dataset and Spectro-Temporal Transformer Model for Monoaural Audio Source Separation

TL;DR

This paper introduces WildMix, a diverse in-the-wild dataset for monoaural source separation, and proposes the Spectro-Temporal Transformer model that effectively captures long-range dependencies for improved separation performance.

Contribution

The paper presents a new challenging dataset WildMix and a novel Spectro-Temporal Transformer model with a specialized encoder for monoaural source separation.

Findings

STT outperforms previous baselines on WildMix

WildMix extends the diversity of in-the-wild audio recordings

Spectro-Temporal Encoder effectively captures temporal and spectral dependencies

Abstract

Monoaural audio source separation is a challenging research area in machine learning. In this area, a mixture containing multiple audio sources is given, and a model is expected to disentangle the mixture into isolated atomic sources. In this paper, we first introduce a challenging new dataset for monoaural source separation called WildMix. WildMix is designed with the goal of extending the boundaries of source separation beyond what previous datasets in this area would allow. It contains diverse in-the-wild recordings from 25 different sound classes, combined with each other using arbitrary composition policies. Source separation often requires modeling long-range dependencies in both temporal and spectral domains. To this end, we introduce a novel trasnformer-based model called Spectro-Temporal Transformer (STT). STT utilizes a specialized encoder, called Spectro-Temporal Encoder (STE). STE highlights temporal and spectral components of sources within a mixture, using a self-attention mechanism. It subsequently disentangles them in a hierarchical manner. In our experiments, STT swiftly outperforms various previous baselines for monoaural source separation on the challenging WildMix dataset.