The 2018 Signal Separation Evaluation Campaign

TL;DR

The 2018 Signal Separation Evaluation Campaign (SiSEC 2018) organized a large-scale audio separation benchmarking effort, introducing new datasets, tools, and reference methods to advance machine-learning based audio separation research.

Contribution

This paper presents the organization of SiSEC 2018, including new datasets, open-source tools, and reference implementations to facilitate audio separation research and benchmarking.

Findings

Participants' results on MUSDB18 dataset

Performance benchmarks for various separation methods

Availability of new tools and reference implementations

Abstract

This paper reports the organization and results for the 2018 community-based Signal Separation Evaluation Campaign (SiSEC 2018). This year's edition was focused on audio and pursued the effort towards scaling up and making it easier to prototype audio separation software in an era of machine-learning based systems. For this purpose, we prepared a new music separation database: MUSDB18, featuring close to 10h of audio. Additionally, open-source software was released to automatically load, process and report performance on MUSDB18. Furthermore, a new official Python version for the BSSEval toolbox was released, along with reference implementations for three oracle separation methods: ideal binary mask, ideal ratio mask, and multichannel Wiener filter. We finally report the results obtained by the participants.