A differentiable short-time Fourier transform with respect to the window length

TL;DR

This paper introduces a differentiable short-time Fourier transform (STFT) that allows the window length to be optimized via gradient descent, enhancing neural network and signal processing applications.

Contribution

The paper presents a theoretical framework for a differentiable STFT with respect to window length, enabling gradient-based optimization of this parameter.

Findings

Differentiable STFT facilitates window length optimization.

Improves neural network performance on signal tasks.

Applicable to various STFT-based algorithms.

Abstract

In this paper, we revisit the use of spectrograms in neural networks, by making the window length a continuous parameter optimizable by gradient descent instead of an empirically tuned integer-valued hyperparameter. The contribution is mostly theoretical at this point, but plugging the modified STFT into any existing neural network is straightforward. We first define a differentiable version of the STFT in the case where local bins centers are fixed and independent of the window length parameter. We then discuss the more difficult case where the window length affects the position and number of bins. We illustrate the benefits of this new tool on an estimation and a classification problems, showing it can be of interest not only to neural networks but to any STFT-based signal processing algorithm.