Iterative autoregression: a novel trick to improve your low-latency speech enhancement model

TL;DR

This paper introduces an innovative autoregressive training method for low-latency streaming speech enhancement models, improving quality by leveraging previous predictions and addressing training-inference mismatch issues.

Contribution

It proposes a simple, effective autoregressive training technique that enhances low-latency speech enhancement models across various architectures and scenarios.

Findings

Stable quality improvements observed across multiple architectures

Effective mitigation of training-inference mismatch issues

Enhanced performance in real-time speech enhancement tasks

Abstract

Streaming models are an essential component of real-time speech enhancement tools. The streaming regime constrains speech enhancement models to use only a tiny context of future information. As a result, the low-latency streaming setup is generally considered a challenging task and has a significant negative impact on the model's quality. However, the sequential nature of streaming generation offers a natural possibility for autoregression, that is, utilizing previous predictions while making current ones. The conventional method for training autoregressive models is teacher forcing, but its primary drawback lies in the training-inference mismatch that can lead to a substantial degradation in quality. In this study, we propose a straightforward yet effective alternative technique for training autoregressive low-latency speech enhancement models. We demonstrate that the proposed approach leads to stable improvement across diverse architectures and training scenarios.