EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems

TL;DR

This paper introduces OD-TTA, an on-demand test-time adaptation framework for embodied visual systems that reduces resource usage on edge devices by activating adaptation only during significant domain shifts.

Contribution

The paper proposes a novel on-demand TTA paradigm with three techniques: domain shift detection, source model selection, and memory-efficient BN updates.

Findings

OD-TTA achieves comparable or better accuracy than existing methods.

It significantly reduces energy and computation overhead.

OD-TTA enables practical deployment of TTA on resource-constrained devices.

Abstract

Continual Test-time adaptation (CTTA) continuously adapts the deployed model on every incoming batch of data. While achieving optimal accuracy, existing CTTA approaches present poor real-world applicability on resource-constrained edge devices, due to the substantial memory overhead and energy consumption. In this work, we first introduce a novel paradigm -- on-demand TTA -- which triggers adaptation only when a significant domain shift is detected. Then, we present OD-TTA, an on-demand TTA framework for accurate and efficient adaptation on edge devices. OD-TTA comprises three innovative techniques: 1) a lightweight domain shift detection mechanism to activate TTA only when it is needed, drastically reducing the overall computation overhead, 2) a source domain selection module that chooses an appropriate source model for adaptation, ensuring high and robust accuracy, 3) a decoupled Batch Normalization (BN) update scheme to enable memory-efficient adaptation with small batch sizes. Extensive experiments show that OD-TTA achieves comparable and even better performance while reducing the energy and computation overhead remarkably, making TTA a practical reality.