Opportunistic and Context-aware Affect Sensing on Smartphones: The Concept, Challenges and Opportunities
Rajib Rana, Margee Hume, John Reilly, Raja Jurdak, Jeffrey Soar

TL;DR
This paper reviews recent advances and challenges in implementing opportunistic, context-aware affect sensing on smartphones, focusing on facial expression and voice detection enabled by new low-power hardware technologies.
Contribution
It identifies key barriers and potential solutions for deploying affect sensing systems that are both opportunistic and context-aware on smartphones.
Findings
Low-power DSP and GPU enable feasible audio-visual affect sensing
Context inference is crucial for accurate affect detection
Barriers include power consumption and contextual complexity
Abstract
Opportunistic affect sensing offers unprecedented potential for capturing spontaneous affect ubiquitously, obviating biases inherent in the laboratory setting. Facial expression and voice are two major affective displays, however most affect sensing systems on smartphone avoid them due to extensive power requirement. Encouragingly, due to the recent advent of low-power DSP (Digital Signal Processing) co-processor and GPU (Graphics Processing Unit) technology, audio and video sensing are becoming more feasible. To properly evaluate opportunistically captured facial expression and voice, contextual information about the dynamic audio-visual stimuli needs to be inferred. This paper discusses recent advances of affect sensing on the smartphone and identifies the key barriers and potential solutions of implementing opportunistic and context-aware affect sensing on smartphone platforms.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEmotion and Mood Recognition · Digital Mental Health Interventions · EEG and Brain-Computer Interfaces
