Predicting Heart Activity from Speech using Data-driven and Knowledge-based features

TL;DR

This paper shows that self-supervised speech models are more effective than traditional acoustic features in predicting heart activity, highlighting the importance of data-driven representations and the need for more speech-based physiological data.

Contribution

It demonstrates the superiority of self-supervised models over acoustic features for heart activity prediction and discusses the impact of individual variability on model performance.

Findings

Self-supervised speech models outperform acoustic features in predicting heart activity.

Individual variability affects model generalizability.

Data-driven representations are valuable for physiological signal prediction.

Abstract

Accurately predicting heart activity and other biological signals is crucial for diagnosis and monitoring. Given that speech is an outcome of multiple physiological systems, a significant body of work studied the acoustic correlates of heart activity. Recently, self-supervised models have excelled in speech-related tasks compared to traditional acoustic methods. However, the robustness of data-driven representations in predicting heart activity remained unexplored. In this study, we demonstrate that self-supervised speech models outperform acoustic features in predicting heart activity parameters. We also emphasize the impact of individual variability on model generalizability. These findings underscore the value of data-driven representations in such tasks and the need for more speech-based physiological data to mitigate speaker-related challenges.