Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detection

TL;DR

This paper introduces frequency dynamic convolution, a novel method that adapts kernels to frequency components in sound event detection, improving accuracy especially for non-stationary sounds with complex time-frequency patterns.

Contribution

The paper proposes frequency dynamic convolution, a new adaptive kernel approach that enhances sound event detection by addressing the limitations of traditional 2D convolution.

Findings

Outperforms baseline by 6.3% in PSDS on DESED dataset

Significantly better than existing content-adaptive methods

More effective for non-stationary sound events with complex patterns

Abstract

2D convolution is widely used in sound event detection (SED) to recognize two dimensional time-frequency patterns of sound events. However, 2D convolution enforces translation equivariance on sound events along both time and frequency axis while frequency is not shift-invariant dimension. In order to improve physical consistency of 2D convolution on SED, we propose frequency dynamic convolution which applies kernel that adapts to frequency components of input. Frequency dynamic convolution outperforms the baseline by 6.3% in DESED validation dataset in terms of polyphonic sound detection score (PSDS). It also significantly outperforms other pre-existing content-adaptive methods on SED. In addition, by comparing class-wise F1 scores of baseline and frequency dynamic convolution, we showed that frequency dynamic convolution is especially more effective for detection of non-stationary sound events with intricate time-frequency patterns. From this result, we verified that frequency dynamic convolution is superior in recognizing frequency-dependent patterns.