BigEAR: Inferring the Ambient and Emotional Correlates from Smartphone-based Acoustic Big Data

TL;DR

This paper introduces BigEAR, a framework that automatically analyzes smartphone-recorded social conversations to infer emotional states, achieving high accuracy and aiding psychological assessment in naturalistic settings.

Contribution

BigEAR is a novel automated acoustic analysis framework that infers emotional states from smartphone data, reducing manual effort and improving accuracy in social and clinical contexts.

Findings

Achieved 88.76% accuracy in emotion inference

Automated analysis reduces manual coding effort

Effective in real-world social scenarios involving health contexts

Abstract

This paper presents a novel BigEAR big data framework that employs psychological audio processing chain (PAPC) to process smartphone-based acoustic big data collected when the user performs social conversations in naturalistic scenarios. The overarching goal of BigEAR is to identify moods of the wearer from various activities such as laughing, singing, crying, arguing, and sighing. These annotations are based on ground truth relevant for psychologists who intend to monitor/infer the social context of individuals coping with breast cancer. We pursued a case study on couples coping with breast cancer to know how the conversations affect emotional and social well being. In the state-of-the-art methods, psychologists and their team have to hear the audio recordings for making these inferences by subjective evaluations that not only are time-consuming and costly, but also demand manual data coding for thousands of audio files. The BigEAR framework automates the audio analysis. We computed the accuracy of BigEAR with respect to the ground truth obtained from a human rater. Our approach yielded overall average accuracy of 88.76% on real-world data from couples coping with breast cancer.