SelfReplay: Adapting Self-Supervised Sensory Models via Adaptive Meta-Task Replay

TL;DR

SelfReplay is a novel framework that personalizes self-supervised sensory models for individual users by employing meta-learning and replay techniques, enabling effective adaptation with minimal data and computational resources.

Contribution

It introduces a self-supervised meta-learning approach combined with user-specific replay for efficient on-device model personalization in heterogeneous domains.

Findings

SelfReplay outperforms baselines with an 8.8% F1-score improvement.

Model adaptation completes within three minutes on a standard smartphone.

The method requires only 9.54% additional memory during adaptation.

Abstract

Self-supervised learning has emerged as a method for utilizing massive unlabeled data for pre-training models, providing an effective feature extractor for various mobile sensing applications. However, when deployed to end-users, these models encounter significant domain shifts attributed to user diversity. We investigate the performance degradation that occurs when self-supervised models are fine-tuned in heterogeneous domains. To address the issue, we propose SelfReplay, a few-shot domain adaptation framework for personalizing self-supervised models. SelfReplay proposes self-supervised meta-learning for initial model pre-training, followed by a user-side model adaptation by replaying the self-supervision with user-specific data. This allows models to adjust their pre-trained representations to the user with only a few samples. Evaluation with four benchmarks demonstrates that SelfReplay outperforms existing baselines by an average F1-score of 8.8%p. Our on-device computational overhead analysis on a commodity off-the-shelf (COTS) smartphone shows that SelfReplay completes adaptation within an unobtrusive latency (in three minutes) with only a 9.54% memory consumption, demonstrating the computational efficiency of the proposed method.