Efficient Neural and Numerical Methods for High-Quality Online Speech Spectrogram Inversion via Gradient Theorem

TL;DR

This paper presents a highly efficient neural and numerical approach for online speech spectrogram inversion that significantly reduces computational costs while maintaining high reconstruction quality.

Contribution

It introduces a small neural network architecture, leverages increased latency, and develops a linear-complexity solver for phase derivative estimation, enabling faster spectrogram inversion.

Findings

Neural network size reduced by 30 times

Inference cost halved with increased latency

Linear solver achieves several orders of magnitude speedup

Abstract

Recent work in online speech spectrogram inversion effectively combines Deep Learning with the Gradient Theorem to predict phase derivatives directly from magnitudes. Then, phases are estimated from their derivatives via least squares, resulting in a high quality reconstruction. In this work, we introduce three innovations that drastically reduce computational cost, while maintaining high quality: Firstly, we introduce a novel neural network architecture with just 8k parameters, 30 times smaller than previous state of the art. Secondly, increasing latency by 1 hop size allows us to further halve the cost of the neural inference step. Thirdly, we we observe that the least squares problem features a tridiagonal matrix and propose a linear-complexity solver for the least squares step that leverages tridiagonality and positive-semidefiniteness, achieving a speedup of several orders of magnitude. We release samples online.