Super-Human Performance in Online Low-latency Recognition of Conversational Speech

TL;DR

This paper presents a low-latency speech recognition system that achieves super-human accuracy (WER of 5.0%) on conversational speech, with only 1 second delay, using innovative attention-based models and incremental inference.

Contribution

The paper introduces a novel low-latency incremental inference method with attention-based models that surpasses human performance in conversational speech recognition.

Findings

Achieves 5.0% WER on Switchboard benchmark.

Operates with only 1 second word-based latency.

Uses multiple attention-based encoder-decoder networks with incremental inference.

Abstract

Achieving super-human performance in recognizing human speech has been a goal for several decades, as researchers have worked on increasingly challenging tasks. In the 1990's it was discovered, that conversational speech between two humans turns out to be considerably more difficult than read speech as hesitations, disfluencies, false starts and sloppy articulation complicate acoustic processing and require robust handling of acoustic, lexical and language context, jointly. Early attempts with statistical models could only reach error rates over 50% and far from human performance (WER of around 5.5%). Neural hybrid models and recent attention-based encoder-decoder models have considerably improved performance as such contexts can now be learned in an integral fashion. However, processing such contexts requires an entire utterance presentation and thus introduces unwanted delays before a recognition result can be output. In this paper, we address performance as well as latency. We present results for a system that can achieve super-human performance (at a WER of 5.0%, over the Switchboard conversational benchmark) at a word based latency of only 1 second behind a speaker's speech. The system uses multiple attention-based encoder-decoder networks integrated within a novel low latency incremental inference approach.