Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform

TL;DR

This paper introduces a differentiable formulation of the short-time Fourier transform (STFT) that allows for gradient-based optimization of its parameters, improving time-frequency analysis and downstream task performance.

Contribution

It presents a unified, differentiable STFT framework that enables joint optimization with neural networks, overcoming traditional heuristic parameter tuning limitations.

Findings

Enhanced time-frequency representations demonstrated on real-world data.

Joint optimization improves downstream task performance.

Outperforms traditional STFT in adaptive signal analysis.

Abstract

The short-time Fourier transform (STFT) is widely used for analyzing non-stationary signals. However, its performance is highly sensitive to its parameters, and manual or heuristic tuning often yields suboptimal results. To overcome this limitation, we propose a unified differentiable formulation of the STFT that enables gradient-based optimization of its parameters. This approach addresses the limitations of traditional STFT parameter tuning methods, which often rely on computationally intensive discrete searches. It enables fine-tuning of the time-frequency representation (TFR) based on any desired criterion. Moreover, our approach integrates seamlessly with neural networks, allowing joint optimization of the STFT parameters and network weights. The efficacy of the proposed differentiable STFT in enhancing TFRs and improving performance in downstream tasks is demonstrated through experiments on both simulated and real-world data.