ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion

TL;DR

This paper introduces ALS-HAR, a wearable ambient light sensor-based human activity recognition system, and explores how to improve its robustness and accuracy by leveraging multi-modal and contrastive learning techniques, despite external light sensitivity.

Contribution

The work presents a novel ALS-based HAR classifier and strategies to enhance environment-invariant accuracy by transferring knowledge from ALS to IMU-based systems.

Findings

ALS-HAR achieves comparable accuracy to existing modalities.

Cross-modal transfer improves IMU-based classifiers' accuracy by up to 4.2%.

ALS can surpass multi-modal sensor fusion in certain scenarios.

Abstract

Despite the widespread integration of ambient light sensors (ALS) in smart devices commonly used for screen brightness adaptation, their application in human activity recognition (HAR), primarily through body-worn ALS, is largely unexplored. In this work, we developed ALS-HAR, a robust wearable light-based motion activity classifier. Although ALS-HAR achieves comparable accuracy to other modalities, its natural sensitivity to external disturbances, such as changes in ambient light, weather conditions, or indoor lighting, makes it challenging for daily use. To address such drawbacks, we introduce strategies to enhance environment-invariant IMU-based activity classifications through augmented multi-modal and contrastive classifications by transferring the knowledge extracted from the ALS. Our experiments on a real-world activity dataset for three different scenarios demonstrate that while ALS-HAR's accuracy strongly relies on external lighting conditions, cross-modal information can still improve other HAR systems, such as IMU-based classifiers.Even in scenarios where ALS performs insufficiently, the additional knowledge enables improved accuracy and macro F1 score by up to 4.2 % and 6.4 %, respectively, for IMU-based classifiers and even surpasses multi-modal sensor fusion models in two of our three experiment scenarios. Our research highlights the untapped potential of ALS integration in advancing sensor-based HAR technology, paving the way for practical and efficient wearable ALS-based activity recognition systems with potential applications in healthcare, sports monitoring, and smart indoor environments.