Detecting Escalation Level from Speech with Transfer Learning and Acoustic-Lexical Information Fusion

TL;DR

This paper presents a novel acoustic-lexical system for speech escalation detection that leverages transfer learning from emotional datasets to improve performance in small data scenarios, achieving significant accuracy gains.

Contribution

It introduces a transfer learning approach combined with acoustic-lexical features and label smoothing for escalation detection from speech.

Findings

Achieved 81.5% UAR on development set, outperforming baseline of 72.2%.

Utilized transfer learning from emotional datasets to enhance escalation detection.

Demonstrated effectiveness of acoustic-lexical fusion and label smoothing in small data settings.

Abstract

Textual escalation detection has been widely applied to e-commerce companies' customer service systems to pre-alert and prevent potential conflicts. Similarly, in public areas such as airports and train stations, where many impersonal conversations frequently take place, acoustic-based escalation detection systems are also useful to enhance passengers' safety and maintain public order. To this end, we introduce a system based on acoustic-lexical features to detect escalation from speech, Voice Activity Detection (VAD) and label smoothing are adopted to further enhance the performance in our experiments. Considering a small set of training and development data, we also employ transfer learning on several wellknown emotional detection datasets, i.e. RAVDESS, CREMA-D, to learn advanced emotional representations that is then applied to the conversational escalation detection task. On the development set, our proposed system achieves 81.5% unweighted average recall (UAR) which significantly outperforms the baseline with 72.2% UAR.