Reverberation Modeling for Source-Filter-based Neural Vocoder

TL;DR

This paper introduces a reverberation module for neural vocoders that models reverberant effects more accurately by estimating room impulse responses, enhancing speech quality in reverberant environments.

Contribution

It proposes two novel approaches for parameterizing and estimating room impulse responses within neural vocoders, improving reverberation modeling and robustness.

Findings

The reverberation module improves the perceived quality of reverberant speech.

UTV-RIR approach is more robust to unknown reverberation conditions.

Joint training with the phase spectrum predictor enhances reverberation effect modeling.

Abstract

This paper presents a reverberation module for source-filter-based neural vocoders that improves the performance of reverberant effect modeling. This module uses the output waveform of neural vocoders as an input and produces a reverberant waveform by convolving the input with a room impulse response (RIR). We propose two approaches to parameterizing and estimating the RIR. The first approach assumes a global time-invariant (GTI) RIR and directly learns the values of the RIR on a training dataset. The second approach assumes an utterance-level time-variant (UTV) RIR, which is invariant within one utterance but varies across utterances, and uses another neural network to predict the RIR values. We add the proposed reverberation module to the phase spectrum predictor (PSP) of a HiNet vocoder and jointly train the model. Experimental results demonstrate that the proposed module was helpful for modeling the reverberation effect and improving the perceived quality of generated reverberant speech. The UTV-RIR was shown to be more robust than the GTI-RIR to unknown reverberation conditions and achieved a perceptually better reverberation effect.