Fully Convolutional Speech Recognition

TL;DR

This paper introduces a fully convolutional neural network approach for speech recognition that directly processes raw waveforms, eliminating traditional feature extraction and achieving state-of-the-art results on benchmark datasets.

Contribution

It presents the first end-to-end convolutional model for speech recognition that operates directly on raw audio, bypassing feature extraction pipelines.

Findings

Matches state-of-the-art on Wall Street Journal

Achieves state-of-the-art on Librispeech among end-to-end models

Outperforms Deep Speech 2 with less data

Abstract

Current state-of-the-art speech recognition systems build on recurrent neural networks for acoustic and/or language modeling, and rely on feature extraction pipelines to extract mel-filterbanks or cepstral coefficients. In this paper we present an alternative approach based solely on convolutional neural networks, leveraging recent advances in acoustic models from the raw waveform and language modeling. This fully convolutional approach is trained end-to-end to predict characters from the raw waveform, removing the feature extraction step altogether. An external convolutional language model is used to decode words. On Wall Street Journal, our model matches the current state-of-the-art. On Librispeech, we report state-of-the-art performance among end-to-end models, including Deep Speech 2 trained with 12 times more acoustic data and significantly more linguistic data.