Polyphonic audio event detection: multi-label or multi-class multi-task classification problem?

TL;DR

This paper proposes a multi-class multi-task approach for polyphonic audio event detection, dividing event categories into groups to better handle overlaps and improve performance over traditional multi-label methods.

Contribution

It introduces a novel multi-class multi-task framework with a specialized network architecture for polyphonic AED, addressing the combinatorial explosion issue.

Findings

Outperforms multi-label approaches on synthetic dataset

Effective handling of high event overlap scenarios

Improved detection accuracy and robustness

Abstract

Polyphonic events are the main error source of audio event detection (AED) systems. In deep-learning context, the most common approach to deal with event overlaps is to treat the AED task as a multi-label classification problem. By doing this, we inherently consider multiple one-vs.-rest classification problems, which are jointly solved by a single (i.e. shared) network. In this work, to better handle polyphonic mixtures, we propose to frame the task as a multi-class classification problem by considering each possible label combination as one class. To circumvent the large number of arising classes due to combinatorial explosion, we divide the event categories into multiple groups and construct a multi-task problem in a divide-and-conquer fashion, where each of the tasks is a multi-class classification problem. A network architecture is then devised for multi-class multi-task modelling. The network is composed of a backbone subnet and multiple task-specific subnets. The task-specific subnets are designed to learn time-frequency and channel attention masks to extract features for the task at hand from the common feature maps learned by the backbone. Experiments on the TUT-SED-Synthetic-2016 with high degree of event overlap show that the proposed approach results in more favorable performance than the common multi-label approach.