Speech-Declipping Transformer with Complex Spectrogram and Learnerble Temporal Features

TL;DR

This paper introduces a transformer-based speech declipping model that combines complex spectrogram analysis with learned temporal features, significantly improving recovery of clipped speech signals across diverse SNR conditions.

Contribution

The novel integration of a TF-transformer with a convolutional temporal feature extractor enhances declipping performance, especially for low-SDR signals, surpassing existing methods.

Findings

Outperforms state-of-the-art declipping models on VoiceBank-DEMAND dataset

Achieves consistent improvements across various SDR levels

Preserves speech quality by avoiding degradation of unclipped portions

Abstract

We present a transformer-based speech-declipping model that effectively recovers clipped signals across a wide range of input signal-to-distortion ratios (SDRs). While recent time-domain deep neural network (DNN)-based declippers have outperformed traditional handcrafted and spectrogram-based DNN approaches, they still struggle with low-SDR inputs. To address this, we incorporate a transformer-based architecture that operates in the time-frequency (TF) domain. The TF-transformer architecture has demonstrated remarkable performance in the speech enhancement task for low-SDR signals but cannot be optimal for the time-domain artifact like clipping. To overcome the limitations of spectrogram-based DNNs, we design an extra convolutional block that directly extracts temporal features from time-domain waveforms. The joint analysis of complex spectrogram and learned temporal features allows the model to improve performance on both high- and low-SDR inputs. Our approach also preserves the unclipped portions of the speech signal during processing, preventing degradation typically seen when only spectral information is used. In evaluations on the VoiceBank-DEMAND and DNS challenge datasets, the proposed model consistently outperformed state-of-the-art (SOTA) declipping models across various metrics, demonstrating its robustness and generalizability.