Learnable Frontends that do not Learn: Quantifying Sensitivity to Filterbank Initialisation

TL;DR

This paper investigates how the initialisation of learnable filterbanks affects their adaptation and performance in audio tasks, revealing high sensitivity to initialisation and limited filter movement during training.

Contribution

It quantifies the impact of initialisation strategies on learnable filterbanks and highlights the need for alternative optimisation methods to improve adaptability.

Findings

Learnable filterbanks are highly sensitive to initialisation.

Limited filter adaptation occurs during training.

Alternative optimisation strategies may enhance performance.

Abstract

While much of modern speech and audio processing relies on deep neural networks trained using fixed audio representations, recent studies suggest great potential in acoustic frontends learnt jointly with a backend. In this study, we focus specifically on learnable filterbanks. Prior studies have reported that in frontends using learnable filterbanks initialised to a mel scale, the learned filters do not differ substantially from their initialisation. Using a Gabor-based filterbank, we investigate the sensitivity of a learnable filterbank to its initialisation using several initialisation strategies on two audio tasks: voice activity detection and bird species identification. We use the Jensen-Shannon Distance and analysis of the learned filters before and after training. We show that although performance is overall improved, the filterbanks exhibit strong sensitivity to their initialisation strategy. The limited movement from initialised values suggests that alternate optimisation strategies may allow a learnable frontend to reach better overall performance.