HeartbeatCam: Self-Triggered Photo Elicitation of Stress Events Using Wearable Sensing

TL;DR

HeartbeatCam is a wearable system that captures contextual images and audio during stress events using physiological signals, aiding mental health treatment by providing detailed post-event data.

Contribution

It introduces a novel wearable sensing system that combines physiological stress detection with open-source AR glasses for contextual data collection during stress events.

Findings

Successfully captures stress-triggering moments with contextual data

Supports collaborative interpretation of stress events with mental health professionals

Enhances recall accuracy of stress episodes for therapy

Abstract

People often recognize what triggered their stress only after the moment has passed. In therapy, this can become a recurring problem: clients are asked to remember what happened between sessions, but the details that matter (where they were, what they saw and heard, what was happening around them) are easy to lose. We introduce HeartbeatCam, a wearable sensing system that gathers contextual information during moments of elevated stress. It uses a consumer smartwatch stress signal to trigger capture from an open-source AR glasses camera, recording a sparse image-audio clip that can later be reviewed and annotated. The system adopts an actionable sensing approach to mental healthcare, using physiological signals along with contextual capture to support collaborative interpretation of stress-triggering moments with mental health professionals.