Leveraging Audio-Tagging Assisted Sound Event Detection using Weakified Strong Labels and Frequency Dynamic Convolutions

TL;DR

This paper introduces a semi-supervised sound event detection method that leverages audio-tagging with weak and strong labels and frequency dynamic convolutions, significantly improving detection performance on the DESED dataset.

Contribution

A novel two-stage semi-supervised framework combining audio-tagging and frequency dynamic convolutions for enhanced sound event detection.

Findings

Outperforms baseline by 45.5% in polyphonic sound detection score

Effectively utilizes weak and pseudo-weak labels for training

Demonstrates significant improvement on DESED dataset

Abstract

Jointly learning from a small labeled set and a larger unlabeled set is an active research topic under semi-supervised learning (SSL). In this paper, we propose a novel SSL method based on a two-stage framework for leveraging a large unlabeled in-domain set. Stage-1 of our proposed framework focuses on audio-tagging (AT), which assists the sound event detection (SED) system in Stage-2. The AT system is trained utilizing a strongly labeled set converted into weak predictions referred to as weakified set, a weakly labeled set, and an unlabeled set. This AT system then infers on the unlabeled set to generate reliable pseudo-weak labels, which are used with the strongly and weakly labeled set to train a frequency dynamic convolutional recurrent neural network-based SED system at Stage-2 in a supervised manner. Our system outperforms the baseline by 45.5% in terms of polyphonic sound detection score on the DESED real validation set.