DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation

TL;DR

DPCore introduces a robust, efficient method for continual test-time adaptation that dynamically manages domain shifts by intelligently updating prompts, significantly improving performance and reducing computational costs.

Contribution

The paper presents DPCore, a novel approach that dynamically adjusts prompts for continual test-time adaptation, addressing domain recurrence and variability issues.

Findings

Outperforms existing CTTA methods across four benchmarks.

Achieves state-of-the-art results in both structured and dynamic domain settings.

Reduces trainable parameters by 99% and computation time by 64%.

Abstract

Continual Test-Time Adaptation (CTTA) seeks to adapt source pre-trained models to continually changing, unseen target domains. While existing CTTA methods assume structured domain changes with uniform durations, real-world environments often exhibit dynamic patterns where domains recur with varying frequencies and durations. Current approaches, which adapt the same parameters across different domains, struggle in such dynamic conditions-they face convergence issues with brief domain exposures, risk forgetting previously learned knowledge, or misapplying it to irrelevant domains. To remedy this, we propose DPCore, a method designed for robust performance across diverse domain change patterns while ensuring computational efficiency. DPCore integrates three key components: Visual Prompt Adaptation for efficient domain alignment, a Prompt Coreset for knowledge preservation, and a Dynamic Update mechanism that intelligently adjusts existing prompts for similar domains while creating new ones for substantially different domains. Extensive experiments on four benchmarks demonstrate that DPCore consistently outperforms various CTTA methods, achieving state-of-the-art performance in both structured and dynamic settings while reducing trainable parameters by 99% and computation time by 64% compared to previous approaches.