Impact of Noisy Labels on Sound Event Detection: Deletion Errors Are More Detrimental Than Insertion Errors

TL;DR

This paper investigates how different types of label noise affect sound event detection, revealing deletion errors are more harmful than insertion errors, and proposes strategies to mitigate noise impact for improved robustness.

Contribution

It systematically analyzes the effects of various label noise types on SED and introduces loss functions tailored to address data imbalance and noise robustness.

Findings

Deletion noise significantly degrades SED performance.

Loss functions for classification noise are ineffective for SED.

Adjusting class weights improves macro-F1 and micro-F1 scores by about 9%.

Abstract

This study explores the critical but underexamined impact of label noise on Sound Event Detection (SED), which requires both sound identification and precise temporal localization. We categorize label noise into deletion, insertion, substitution, and subjective types and systematically evaluate their effects on SED using synthetic and real-life datasets. Our analysis shows that deletion noise significantly degrades performance, while insertion noise is relatively benign. Moreover, loss functions effective against classification noise do not perform well for SED due to intra-class imbalance between foreground sound events and background sounds. We demonstrate that loss functions designed to address data imbalance in SED can effectively reduce the impact of noisy labels on system performance. For instance, halving the weight of background sounds in a synthetic dataset improved macro-F1 and micro-F1 scores by approximately $9\%$ with minimal Error Rate increase, with consistent results in real-life datasets. This research highlights the nuanced effects of noisy labels on SED systems and provides practical strategies to enhance model robustness, which are pivotal for both constructing new SED datasets and improving model performance, including efficient utilization of soft and crowdsourced labels.