Insights on Modelling Physiological, Appraisal, and Affective Indicators of Stress using Audio Features

TL;DR

This paper investigates how speech features can be used to model physiological, appraisal, and affective stress indicators, revealing that multi-task learning enhances prediction accuracy across these diverse stress-related measures.

Contribution

It introduces a multi-task learning approach that effectively combines different stress indicators from speech signals, highlighting their complementary information.

Findings

Speech features can model cortisol, appraisal, and affect measures.

Multi-task architecture improves prediction accuracy.

Different indicators influence acoustic features diversely.

Abstract

Stress is a major threat to well-being that manifests in a variety of physiological and mental symptoms. Utilising speech samples collected while the subject is undergoing an induced stress episode has recently shown promising results for the automatic characterisation of individual stress responses. In this work, we introduce new findings that shed light onto whether speech signals are suited to model physiological biomarkers, as obtained via cortisol measurements, or self-assessed appraisal and affect measurements. Our results show that different indicators impact acoustic features in a diverse way, but that their complimentary information can nevertheless be effectively harnessed by a multi-tasking architecture to improve prediction performance for all of them.