Should Audio Front-ends be Adaptive? Comparing Learnable and Adaptive Front-ends

TL;DR

This paper compares adaptive and learnable audio front-ends, demonstrating that the adaptive Ada-FE outperforms learnable alternatives in accuracy and robustness across multiple audio benchmarks.

Contribution

It introduces and systematically evaluates the adaptive Ada-FE front-end, showing its advantages over existing learnable front-ends in diverse audio tasks.

Findings

Ada-FE outperforms learnable front-ends in accuracy.

Ada-FE demonstrates greater robustness over training epochs.

Comprehensive benchmarks validate Ada-FE's effectiveness.

Abstract

Hand-crafted features, such as Mel-filterbanks, have traditionally been the choice for many audio processing applications. Recently, there has been a growing interest in learnable front-ends that extract representations directly from the raw audio waveform. \textcolor{black}{However, both hand-crafted filterbanks and current learnable front-ends lead to fixed computation graphs at inference time, failing to dynamically adapt to varying acoustic environments, a key feature of human auditory systems.} To this end, we explore the question of whether audio front-ends should be adaptive by comparing the Ada-FE front-end (a recently developed adaptive front-end that employs a neural adaptive feedback controller to dynamically adjust the Q-factors of its spectral decomposition filters) to established learnable front-ends. Specifically, we systematically investigate learnable front-ends and Ada-FE across two commonly used back-end backbones and a wide range of audio benchmarks including speech, sound event, and music. The comprehensive results show that our Ada-FE outperforms advanced learnable front-ends, and more importantly, it exhibits impressive stability or robustness on test samples over various training epochs.