SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data

TL;DR

SelfHAR is a semi-supervised learning framework that leverages unlabeled mobile sensing data through self-training and self-supervision, significantly improving human activity recognition accuracy and data efficiency.

Contribution

The paper introduces SelfHAR, a novel semi-supervised model combining teacher-student self-training and multi-task self-supervision for HAR, achieving state-of-the-art results with less labeled data.

Findings

Up to 12% increase in F1 score over previous methods.

Achieves similar performance with 10 times less labeled data.

State-of-the-art performance across multiple HAR datasets.

Abstract

Machine learning and deep learning have shown great promise in mobile sensing applications, including Human Activity Recognition. However, the performance of such models in real-world settings largely depends on the availability of large datasets that captures diverse behaviors. Recently, studies in computer vision and natural language processing have shown that leveraging massive amounts of unlabeled data enables performance on par with state-of-the-art supervised models. In this work, we present SelfHAR, a semi-supervised model that effectively learns to leverage unlabeled mobile sensing datasets to complement small labeled datasets. Our approach combines teacher-student self-training, which distills the knowledge of unlabeled and labeled datasets while allowing for data augmentation, and multi-task self-supervision, which learns robust signal-level representations by predicting distorted versions of the input. We evaluated SelfHAR on various HAR datasets and showed state-of-the-art performance over supervised and previous semi-supervised approaches, with up to 12% increase in F1 score using the same number of model parameters at inference. Furthermore, SelfHAR is data-efficient, reaching similar performance using up to 10 times less labeled data compared to supervised approaches. Our work not only achieves state-of-the-art performance in a diverse set of HAR datasets, but also sheds light on how pre-training tasks may affect downstream performance.