AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments

TL;DR

AdaShadow is a novel framework enabling rapid, resource-aware test-time model adaptation on mobile devices, effectively handling non-stationary data with minimal latency and improved accuracy.

Contribution

It introduces a backpropagation-free critical layer assessor, a runtime resource predictor, and an online scheduler for efficient, responsive adaptation in mobile environments.

Findings

Achieves 2x to 3.5x speedup over state-of-the-art TTA methods.

Provides 14.8% to 25.4% accuracy boost over efficient supervised methods.

Reduces latency with a memory I/O-aware computation reuse scheme.

Abstract

On-device adapting to continual, unpredictable domain shifts is essential for mobile applications like autonomous driving and augmented reality to deliver seamless user experiences in evolving environments. Test-time adaptation (TTA) emerges as a promising solution by tuning model parameters with unlabeled live data immediately before prediction. However, TTA's unique forward-backward-reforward pipeline notably increases the latency over standard inference, undermining the responsiveness in time-sensitive mobile applications. This paper presents AdaShadow, a responsive test-time adaptation framework for non-stationary mobile data distribution and resource dynamics via selective updates of adaptation-critical layers. Although the tactic is recognized in generic on-device training, TTA's unsupervised and online context presents unique challenges in estimating layer importance and latency, as well as scheduling the optimal layer update plan. AdaShadow addresses these challenges with a backpropagation-free assessor to rapidly identify critical layers, a unit-based runtime predictor to account for resource dynamics in latency estimation, and an online scheduler for prompt layer update planning. Also, AdaShadow incorporates a memory I/O-aware computation reuse scheme to further reduce latency in the reforward pass. Results show that AdaShadow achieves the best accuracy-latency balance under continual shifts. At low memory and energy costs, Adashadow provides a 2x to 3.5x speedup (ms-level) over state-of-the-art TTA methods with comparable accuracy and a 14.8% to 25.4% accuracy boost over efficient supervised methods with similar latency.