SocialPulse: An On-Smartwatch System for Detecting Real-World Social Interactions

TL;DR

SocialPulse is a smartwatch system that unobtrusively detects real-world social interactions, including in-person and virtual, using transfer learning and conversational cues, achieving over 73% accuracy in real-world tests.

Contribution

We introduce a real-time smartwatch system capable of detecting diverse social interactions in daily life, overcoming limitations of previous controlled-environment systems.

Findings

Achieved 73.18% interaction detection accuracy in real-world evaluation.

Perfect recall in follow-up tests for detecting interactions.

Demonstrated potential for social behavior monitoring and interventions.

Abstract

Social interactions are a fundamental part of daily life and play a critical role in well-being. As emerging technologies offer opportunities to unobtrusively monitor behavior, there is growing interest in using them to better understand social experiences. However, automatically detecting interactions, particularly via wearable devices, remains underexplored. Existing systems are often limited to controlled environments, constrained to in-person interactions, and rely on rigid assumptions such as the presence of two speakers within a fixed time window. These limitations reduce their generalizability to capture diverse real-world interactions. To address these challenges, we developed a real-time, on-watch system capable of detecting both in-person and virtual interactions. The system leverages transfer learning to detect foreground speech (FS) and infers interaction boundaries based upon FS and conversational cues like whispering. In a real-world evaluation involving 11 participants over a total of 38 days (Mean = 3.45 days, SD = 2.73), the system achieved an interaction detection accuracy of 73.18%. Follow-up with six participants indicated perfect recall for detecting interactions. These preliminary findings demonstrate the potential of our system to capture interactions in daily life, providing a foundation for applications such as personalized interventions targeting social anxiety.