A Mixed supervised Learning Framework for Target Sound Detection

TL;DR

This paper introduces a mixed supervised learning framework with two mutually-helping models to improve target sound detection by leveraging both fully-annotated and weakly-annotated data, achieving significant performance gains.

Contribution

It proposes a novel dual-student model approach that iteratively enhances target sound detection performance using mixed supervision from different annotation levels.

Findings

Achieved about 8% improvement in event-based F-score over baselines.

Demonstrated effectiveness on UrbanSound and Audioset datasets.

Validated the framework's ability to leverage mixed supervision for better detection.

Abstract

Target sound detection (TSD) aims to detect the target sound from mixture audio given the reference information. Previous works have shown that TSD models can be trained on fully-annotated (frame-level label) or weakly-annotated (clip-level label) data. However, there are some clear evidences show that the performance of the model trained on weakly-annotated data is worse than that trained on fully-annotated data. To fill this gap, we provide a mixed supervision perspective, in which learning novel categories (target domain) using weak annotations with the help of full annotations of existing base categories (source domain). To realize this, a mixed supervised learning framework is proposed, which contains two mutually-helping student models (\textit{f\_student} and \textit{w\_student}) that learn from fully-annotated and weakly-annotated data, respectively. The motivation is that \textit{f\_student} learned from fully-annotated data has a better ability to capture detailed information than \textit{w\_student}. Thus, we first let \textit{f\_student} guide \textit{w\_student} to learn the ability to capture details, so \textit{w\_student} can perform better in the target domain. Then we let \textit{w\_student} guide \textit{f\_student} to fine-tune on the target domain. The process can be repeated several times so that the two students perform very well in the target domain. To evaluate our method, we built three TSD datasets based on UrbanSound and Audioset. Experimental results show that our methods offer about 8\% improvement in event-based F-score as compared with a recent baseline.