Recognizing Detailed Human Context In-the-Wild from Smartphones and Smartwatches

TL;DR

This paper presents a large-scale, real-world dataset and a baseline system for recognizing detailed human context from smartphone and smartwatch sensors, emphasizing the importance of multi-modal data fusion in unconstrained environments.

Contribution

It introduces a novel, extensive in-the-wild dataset with naturalistic sensor data and labels, and provides a baseline system to advance practical context recognition research.

Findings

Multi-modal sensor fusion improves recognition accuracy.

Unscripted, natural behavior increases recognition difficulty.

Public dataset enables benchmarking and progress in real-world context recognition.

Abstract

The ability to automatically recognize a person's behavioral context can contribute to health monitoring, aging care and many other domains. Validating context recognition in-the-wild is crucial to promote practical applications that work in real-life settings. We collected over 300k minutes of sensor data with context labels from 60 subjects. Unlike previous studies, our subjects used their own personal phone, in any way that was convenient to them, and engaged in their routine in their natural environments. Unscripted behavior and unconstrained phone usage resulted in situations that are harder to recognize. We demonstrate how fusion of multi-modal sensors is important for resolving such cases. We present a baseline system, and encourage researchers to use our public dataset to compare methods and improve context recognition in-the-wild.