Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization

TL;DR

This paper introduces a CNN-Transformer model for weakly labeled sound event detection and proposes an automatic threshold optimization method, significantly improving detection and tagging performance over previous approaches.

Contribution

It presents a novel CNN-Transformer architecture for audio tagging and SED, along with an automatic threshold optimization method to enhance detection accuracy.

Findings

Achieved state-of-the-art F1 scores for audio tagging and SED.

Demonstrated CNN-Transformer performs comparably to CRNN.

Automatic threshold optimization improves F1 scores significantly.

Abstract

Sound event detection (SED) is a task to detect sound events in an audio recording. One challenge of the SED task is that many datasets such as the Detection and Classification of Acoustic Scenes and Events (DCASE) datasets are weakly labelled. That is, there are only audio tags for each audio clip without the onset and offset times of sound events. \qk{We compare segment-wise and clip-wise training for SED that is lacking in previous works. We propose a convolutional neural network transformer (CNN-Transfomer) for audio tagging and SED, and show that CNN-Transformer performs similarly to a convolutional recurrent neural network (CRNN)}. Another challenge of SED is that thresholds are required for detecting sound events. Previous works set thresholds empirically, and are not an optimal approaches. To solve this problem, we propose an automatic threshold optimization method. The first stage is to optimize the system with respect to metrics that do not depend on thresholds, such as mean average precision (mAP). The second stage is to optimize the thresholds with respect to metrics that depends on those thresholds. Our proposed automatic threshold optimization system achieves a state-of-the-art audio tagging F1 of 0.646, outperforming that without threshold optimization of 0.629, and a sound event detection F1 of 0.584, outperforming that without threshold optimization of 0.564.