Temporal Structure Matters for Efficient Test-Time Adaptation in Wearable Human Activity Recognition

TL;DR

This paper introduces SIGHT, a lightweight test-time adaptation framework for wearable human activity recognition that leverages temporal structure and feature deviations to improve real-time model adaptation.

Contribution

It proposes a novel, backpropagation-free TTA method that uses temporal cues and feature deviations for efficient, real-time adaptation in wearable activity recognition.

Findings

SIGHT outperforms existing TTA methods on real-world datasets.

SIGHT reduces computational and memory costs compared to baselines.

SIGHT effectively leverages temporal structure for better adaptation.

Abstract

Wearable human activity recognition (WHAR) models often suffer from performance degradation under real-world cross-user distribution shifts. Test-time adaptation (TTA) mitigates this degradation by adapting models online using unlabeled test streams, yet existing methods largely inherit assumptions from vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams. In this paper, we revisit such temporal structure as a feature-conditioned inference signal rather than merely an output-space smoothing prior. We derive the insight that temporal continuity and observation-induced feature deviations provide complementary cues for determining when to preserve or release temporal inertia and where to route prediction refinement during likely transitions. Building upon this insight, we propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment. SIGHT estimates predictive surprise by comparing the current feature with a prototype-based expected state, and then uses the resulting feature deviation to guide geometry-aware transition routing based on prototype alignment and stream-level marginal habit tracking. Evaluations on real-world datasets confirm that SIGHT outperforms existing TTA baselines while reducing computational and memory costs.