Detecting Sound Events Using Convolutional Macaron Net With Pseudo Strong Labels

TL;DR

This paper introduces a semi-supervised approach for sound event detection using pseudo labels generated by Nonnegative Matrix Factorization and a novel Convolutional Macaron Net architecture trained with dual models and curriculum consistency, achieving state-of-the-art results.

Contribution

The paper presents a new semi-supervised framework combining pseudo labeling and a dual-model training scheme with curriculum consistency for sound event detection.

Findings

Outperforms DCASE 2020 baseline by over 10%

Achieves higher accuracy than DCASE 2019 top submission by 1.8%

Maintains robustness on unseen YouTube data

Abstract

In this paper, we propose addressing the lack of strongly labeled data by using pseudo strongly labeled data approximated using Convolutive Nonnegative Matrix Factorization. Using this set of data, we then train a novel architecture called the Convolutional Macaron Net (CMN), which combines Convolutional Neural Network (CNN) with MN, in a semi-supervised manner. Instead of training only a single model or using the Mean-teacher approach, we train two different CMNs synchronously using a curriculum consistency cost and a curriculum interpolated consistency cost. In the inference stage, one of the models will provide the frame-level prediction while the other model will provide the clip-level prediction. Our system outperforms the baseline system of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 Challenge Task 4 by a margin of over 10% based on our proposed framework. By comparing with the top submission of the DCASE 2019 challenge, our system accuracy is also higher by 1.8%. On the other hand, as compared to the top submission of DCASE 2020, our accuracy is also marginally higher by 0.3%, even with fewer Transformer encoding layers. Our system remains robust on unseen YouTube evaluation dataset and has a winning margin of 0.6% and 6.3% against the top submission of DCASE 2019 and the baseline system.