An Experimental Study on Joint Modeling for Sound Event Localization and Detection with Source Distance Estimation

TL;DR

This paper introduces three innovative joint modeling approaches for 3D sound event localization and detection, including source distance estimation, significantly advancing spatial sound analysis.

Contribution

It presents novel methods for integrating source distance estimation with localization and detection, achieving state-of-the-art results in 3D SELD tasks.

Findings

Ranked first in the DCASE 2024 Challenge Task 3

Demonstrated effectiveness of joint modeling approaches

Proposed methods outperform traditional separate models

Abstract

In traditional sound event localization and detection (SELD) tasks, the focus is typically on sound event detection (SED) and direction-of-arrival (DOA) estimation, but they fall short of providing full spatial information about the sound source. The 3D SELD task addresses this limitation by integrating source distance estimation (SDE), allowing for complete spatial localization. We propose three approaches to tackle this challenge: a novel method with independent training and joint prediction, which firstly treats DOA and distance estimation as separate tasks and then combines them to solve 3D SELD; a dual-branch representation with source Cartesian coordinate used for simultaneous DOA and distance estimation; and a three-branch structure that jointly models SED, DOA, and SDE within a unified framework. Our proposed method ranked first in the DCASE 2024 Challenge Task 3, demonstrating the effectiveness of joint modeling for addressing the 3D SELD task. The relevant code for this paper will be open-sourced in the future.