Processing Self Corrections in a speech to speech system

TL;DR

This paper introduces a multi-level framework for detecting and correcting speech repairs in spontaneous dialogues by cascading filters across acoustics, lexis, syntax, and semantics to improve spoken language systems.

Contribution

It presents a novel integrated approach combining acoustic, lexical, syntactic, and semantic information with cascading filters for effective speech repair correction.

Findings

Improved detection of speech repairs in spontaneous speech.

Enhanced correction accuracy through multi-level filtering.

Integration of acoustic and linguistic features boosts system robustness.

Abstract

Speech repairs occur often in spontaneous spoken dialogues. The ability to detect and correct those repairs is necessary for any spoken language system. We present a framework to detect and correct speech repairs where all relevant levels of information, i.e., acoustics, lexis, syntax and semantics can be integrated. The basic idea is to reduce the search space for repairs as soon as possible by cascading filters that involve more and more features. At first an acoustic module generates hypotheses about the existence of a repair. Second a stochastic model suggests a correction for every hypothesis. Well scored corrections are inserted as new paths in the word lattice. Finally a lattice parser decides on accepting the rep air.