Improving Device Directedness Classification of Utterances with Semantic Lexical Features

TL;DR

This paper introduces a novel device-directedness classifier combining semantic lexical and acoustic features, achieving significant improvements over existing acoustic-only models and further gains through transfer and semi-supervised learning.

Contribution

It presents a new mixed-domain feature model for classifying utterance directedness, outperforming state-of-the-art acoustic-only systems and incorporating transfer and semi-supervised learning techniques.

Findings

14% relative reduction in EER over baseline

Effective combination of semantic lexical and acoustic features

Enhanced accuracy via transfer and semi-supervised learning

Abstract

User interactions with personal assistants like Alexa, Google Home and Siri are typically initiated by a wake term or wakeword. Several personal assistants feature "follow-up" modes that allow users to make additional interactions without the need of a wakeword. For the system to only respond when appropriate, and to ignore speech not intended for it, utterances must be classified as device-directed or non-device-directed. State-of-the-art systems have largely used acoustic features for this task, while others have used only lexical features or have added LM-based lexical features. We propose a directedness classifier that combines semantic lexical features with a lightweight acoustic feature and show it is effective in classifying directedness. The mixed-domain lexical and acoustic feature model is able to achieve 14% relative reduction of EER over a state-of-the-art acoustic-only baseline model. Finally, we successfully apply transfer learning and semi-supervised learning to the model to improve accuracy even further.