Exploring Text-Queried Sound Event Detection with Audio Source Separation

TL;DR

This paper introduces a text-queried sound event detection framework that leverages a pre-trained language-queried source separation model, enhanced with a dual-path RNN, to improve detection accuracy in overlapping sound scenarios.

Contribution

It proposes a novel TQ-SED framework combining language-queried source separation with a dual-path RNN, achieving state-of-the-art results in language-queried audio source separation.

Findings

TQ-SED improves F1 score by 7.22% over conventional methods.

AudioSep-DP achieves first place in DCASE 2024 Task 9.

Enhanced model complexity impacts separation performance.

Abstract

In sound event detection (SED), overlapping sound events pose a significant challenge, as certain events can be easily masked by background noise or other events, resulting in poor detection performance. To address this issue, we propose the text-queried SED (TQ-SED) framework. Specifically, we first pre-train a language-queried audio source separation (LASS) model to separate the audio tracks corresponding to different events from the input audio. Then, multiple target SED branches are employed to detect individual events. AudioSep is a state-of-the-art LASS model, but has limitations in extracting dynamic audio information because of its pure convolutional structure for separation. To address this, we integrate a dual-path recurrent neural network block into the model. We refer to this structure as AudioSep-DP, which achieves the first place in DCASE 2024 Task 9 on language-queried audio source separation (objective single model track). Experimental results show that TQ-SED can significantly improve the SED performance, with an improvement of 7.22\% on F1 score over the conventional framework. Additionally, we setup comprehensive experiments to explore the impact of model complexity. The source code and pre-trained model are released at https://github.com/apple-yinhan/TQ-SED.