A General Network Architecture for Sound Event Localization and Detection Using Transfer Learning and Recurrent Neural Network

TL;DR

This paper introduces a versatile neural network architecture for sound event localization and detection that combines independently pretrained sub-networks for detection and localization with a recurrent layer, demonstrating competitive results on benchmark datasets.

Contribution

The proposed architecture allows independent training of SED and DOA sub-networks and integrates them with a recurrent layer, enhancing flexibility and performance in SELD tasks.

Findings

Competitive performance on DCASE 2020 dataset

Compatible with various SED and DOA algorithms

Flexible architecture for independent sub-network improvement

Abstract

Polyphonic sound event detection and localization (SELD) task is challenging because it is difficult to jointly optimize sound event detection (SED) and direction-of-arrival (DOA) estimation in the same network. We propose a general network architecture for SELD in which the SELD network comprises sub-networks that are pretrained to solve SED and DOA estimation independently, and a recurrent layer that combines the SED and DOA estimation outputs into SELD outputs. The recurrent layer does the alignment between the sound classes and DOAs of sound events while being unaware of how these outputs are produced by the upstream SED and DOA estimation algorithms. This simple network architecture is compatible with different existing SED and DOA estimation algorithms. It is highly practical since the sub-networks can be improved independently. The experimental results using the DCASE 2020 SELD dataset show that the performances of our proposed network architecture using different SED and DOA estimation algorithms and different audio formats are competitive with other state-of-the-art SELD algorithms. The source code for the proposed SELD network architecture is available at Github.