A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4

TL;DR

This paper presents a hybrid sound event detection system combining a transformer-based model and a frame-wise CNN, leveraging semi-supervised learning to improve detection accuracy for DCASE 2022 Task 4.

Contribution

It introduces a novel hybrid system that integrates an end-to-end transformer model with a frame-wise CNN, utilizing semi-supervised learning for enhanced sound event detection.

Findings

Hybrid system outperforms individual models.

Achieves psds1 of 0.420 and psds2 of 0.783 on validation set.

Utilizes self-supervised pre-training and semi-supervised learning.

Abstract

In this paper, we describe in detail our system for DCASE 2022 Task4. The system combines two considerably different models: an end-to-end Sound Event Detection Transformer (SEDT) and a frame-wise model, Metric Learning and Focal Loss CNN (MLFL-CNN). The former is an event-wise model which learns event-level representations and predicts sound event categories and boundaries directly, while the latter is based on the widely adopted frame-classification scheme, under which each frame is classified into event categories and event boundaries are obtained by post-processing such as thresholding and smoothing. For SEDT, self-supervised pre-training using unlabeled data is applied, and semi-supervised learning is adopted by using an online teacher, which is updated from the student model using the Exponential Moving Average (EMA) strategy and generates reliable pseudo labels for weakly-labeled and unlabeled data. For the frame-wise model, the ICT-TOSHIBA system of DCASE 2021 Task 4 is used. Experimental results show that the hybrid system considerably outperforms either individual model and achieves psds1 of 0.420 and psds2 of 0.783 on the validation set without external data. The code is available at https://github.com/965694547/Hybrid-system-of-frame-wise-model-and-SEDT.