Towards Open World Sound Event Detection

TL;DR

This paper introduces OW-SED, a new paradigm for sound event detection that handles known, unknown, and incremental learning of acoustic events using a deformable transformer architecture.

Contribution

It proposes a novel OW-SED framework with a deformable attention-based architecture and a disentangled representation approach for open-world sound event detection.

Findings

Achieves marginally better performance in closed-world settings.

Significantly outperforms baselines in open-world scenarios.

Demonstrates effectiveness of deformable attention and feature disentanglement.

Abstract

Sound Event Detection (SED) plays a vital role in audio understanding, with applications in surveillance, smart cities, healthcare, and multimedia indexing. However, conventional SED systems operate under a closed-world assumption, limiting their effectiveness in real-world environments where novel acoustic events frequently emerge. Inspired by the success of open-world learning in computer vision, we introduce the Open-World Sound Event Detection (OW-SED) paradigm, where models must detect known events, identify unseen ones, and incrementally learn from them. To tackle the unique challenges of OW-SED, such as overlapping and ambiguous events, we propose a 1D Deformable architecture that leverages deformable attention to adaptively focus on salient temporal regions. Furthermore, we design a novel Open-World Deformable Sound Event Detection Transformer (WOOT) framework incorporating feature disentanglement to separate class-specific and class-agnostic representations, together with a one-to-many matching strategy and a diversity loss to enhance representation diversity. Experimental results demonstrate that our method achieves marginally superior performance compared to existing leading techniques in closed-world settings and significantly improves over existing baselines in open-world scenarios.