Letter-Based Speech Recognition with Gated ConvNets

TL;DR

This paper introduces a letter-based speech recognition system using Gated ConvNets, achieving competitive results on WSJ and LibriSpeech benchmarks without relying on traditional phoneme-based models.

Contribution

It presents a novel ConvNet architecture with Gated Linear Units for letter-based speech recognition, demonstrating strong performance with CTC and ASG training methods.

Findings

Matches best letter-based systems on WSJ

Achieves near state-of-the-art on LibriSpeech

Utilizes Gated ConvNets with high dropout

Abstract

In the recent literature, "end-to-end" speech systems often refer to letter-based acoustic models trained in a sequence-to-sequence manner, either via a recurrent model or via a structured output learning approach (such as CTC). In contrast to traditional phone (or senone)-based approaches, these "end-to-end'' approaches alleviate the need of word pronunciation modeling, and do not require a "forced alignment" step at training time. Phone-based approaches remain however state of the art on classical benchmarks. In this paper, we propose a letter-based speech recognition system, leveraging a ConvNet acoustic model. Key ingredients of the ConvNet are Gated Linear Units and high dropout. The ConvNet is trained to map audio sequences to their corresponding letter transcriptions, either via a classical CTC approach, or via a recent variant called ASG. Coupled with a simple decoder at inference time, our system matches the best existing letter-based systems on WSJ (in word error rate), and shows near state of the art performance on LibriSpeech.