DySTAN: Joint Modeling of Sedentary Activity and Social Context from Smartphone Sensors

TL;DR

This paper introduces DySTAN, a multi-task learning framework that jointly classifies sedentary activity and social context from smartphone sensors, significantly improving recognition accuracy by modeling co-occurring contexts.

Contribution

The paper presents DySTAN, a novel multi-task learning model that effectively captures subtle differences in sedentary and social contexts from shared sensor data, advancing mobile context recognition.

Findings

DySTAN improves sedentary activity macro F1 scores by 21.8%.

DySTAN outperforms single-task and baseline multi-task models.

Joint modeling of multiple contexts enhances recognition robustness.

Abstract

Accurately recognizing human context from smartphone sensor data remains a significant challenge, especially in sedentary settings where activities such as studying, attending lectures, relaxing, and eating exhibit highly similar inertial patterns. Furthermore, social context plays a critical role in understanding user behavior, yet is often overlooked in mobile sensing research. To address these gaps, we introduce LogMe, a mobile sensing application that passively collects smartphone sensor data (accelerometer, gyroscope, magnetometer, and rotation vector) and prompts users for hourly self-reports capturing both sedentary activity and social context. Using this dual-label dataset, we propose DySTAN (Dynamic Cross-Stitch with Task Attention Network), a multi-task learning framework that jointly classifies both context dimensions from shared sensor inputs. It integrates task-specific layers with cross-task attention to model subtle distinctions effectively. DySTAN improves sedentary activity macro F1 scores by 21.8% over a single-task CNN-BiLSTM-GRU (CBG) model and by 8.2% over the strongest multi-task baseline, Sluice Network (SN). These results demonstrate the importance of modeling multiple, co-occurring context dimensions to improve the accuracy and robustness of mobile context recognition.