Fitting Auditory Filterbanks with Multiresolution Neural Networks

TL;DR

This paper introduces MuReNN, a multiresolution neural network that combines wavelet transforms with convolutional learning to accurately model auditory filterbanks, improving time-frequency localization and fitting real-world data.

Contribution

MuReNN integrates wavelet-based multiresolution analysis with neural networks, overcoming limitations of purely parametric or nonparametric models in auditory filterbank approximation.

Findings

MuReNN achieves state-of-the-art fit to Gammatone, CQT, and third-octave filterbanks.

It improves time-frequency localization compared to traditional convnets.

MuReNN effectively combines domain knowledge with data-driven learning.

Abstract

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is guaranteed to yield Gabor filters, hence an optimal time-frequency localization; yet, this strong inductive bias comes at the detriment of representational capacity. In this paper, we aim to overcome this dilemma by introducing a neural audio model, named multiresolution neural network (MuReNN). The key idea behind MuReNN is to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT). Since the scale of DWT atoms grows exponentially between octaves, the receptive fields of the subsequent learnable convolutions in MuReNN are dilated accordingly. For a given real-world dataset, we fit the magnitude response of MuReNN to that of a well-established auditory filterbank: Gammatone for speech, CQT for music, and third-octave for urban sounds, respectively. This is a form of knowledge distillation (KD), in which the filterbank ''teacher'' is engineered by domain knowledge while the neural network ''student'' is optimized from data. We compare MuReNN to the state of the art in terms of goodness of fit after KD on a hold-out set and in terms of Heisenberg time-frequency localization. Compared to convnets and Gabor convolutions, we find that MuReNN reaches state-of-the-art performance on all three optimization problems.