Deep Speech: Scaling up end-to-end speech recognition

TL;DR

Deep Speech is an end-to-end deep learning speech recognition system that simplifies traditional pipelines, learns directly from data, and outperforms previous models especially in noisy environments, using novel training and data synthesis techniques.

Contribution

The paper introduces a simplified end-to-end speech recognition system that eliminates the need for hand-engineered components and phoneme dictionaries, achieving superior accuracy and noise robustness.

Findings

Achieves 16.0% error on Switchboard Hub5'00 test set.

Outperforms previous models in noisy environments.

Uses novel data synthesis for training.

Abstract

We present a state-of-the-art speech recognition system developed using end-to-end deep learning. Our architecture is significantly simpler than traditional speech systems, which rely on laboriously engineered processing pipelines; these traditional systems also tend to perform poorly when used in noisy environments. In contrast, our system does not need hand-designed components to model background noise, reverberation, or speaker variation, but instead directly learns a function that is robust to such effects. We do not need a phoneme dictionary, nor even the concept of a "phoneme." Key to our approach is a well-optimized RNN training system that uses multiple GPUs, as well as a set of novel data synthesis techniques that allow us to efficiently obtain a large amount of varied data for training. Our system, called Deep Speech, outperforms previously published results on the widely studied Switchboard Hub5'00, achieving 16.0% error on the full test set. Deep Speech also handles challenging noisy environments better than widely used, state-of-the-art commercial speech systems.