Exploring Workplace Behaviors through Speaking Patterns using Large-scale Multimodal Wearable Recordings: A Study of Healthcare Providers

TL;DR

This study uses wearable multimodal sensors to analyze speaking and non-verbal behaviors of healthcare providers, revealing how these patterns relate to work schedules, physiological states, and well-being over time.

Contribution

It introduces a large-scale multimodal data collection approach to study interpersonal communication behaviors among healthcare workers, linking speaking patterns with physiological and psychological measures.

Findings

Speaking patterns vary across shifts and units.

Combined speaking and physiological data predict affect and life satisfaction.

Longitudinal analysis reveals behavioral differences over time.

Abstract

Interpersonal spoken communication is central to human interaction and the exchange of information. Such interactive processes involve not only speech and spoken language but also non-verbal cues such as hand gestures, facial expressions, and nonverbal vocalization, that are used to express feelings and provide feedback. These multimodal communication signals carry a variety of information about the people: traits like gender and age as well as about physical and psychological states and behavior. This work uses wearable multimodal sensors to investigate interpersonal communication behaviors focusing on speaking patterns among healthcare providers with a focus on nurses. We analyze longitudinal data collected from $99$ nurses in a large hospital setting over ten weeks. The results indicate that speaking pattern differences across shift schedules and working units. Moreover, results show that speaking patterns combined with physiological measures can be used to predict affect measures and life satisfaction scores. The implementation of this work can be accessed at https://github.com/usc-sail/tiles-audio-arousal.