Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

TL;DR

This paper introduces RMemSafe, a reliability-gated method for continual test-time adaptation that dynamically reduces reliance on source anchors when source reliability diminishes, improving robustness on corrupted data.

Contribution

It proposes a novel entropy-based gating mechanism for source anchoring in CTTA, enhancing adaptation robustness under source degradation.

Findings

RMemSafe achieves lowest error on 8 of 9 continual-corruption tasks.

It improves ROID+ASR by 1.05 percentage points on ResNet-50.

The entropy gate effectively detects source collapse, enabling graceful adaptation.

Abstract

Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a frozen source checkpoint. This anchor is useful only when the source remains reliable. On CCC-Hard, however, a ResNet-50 source falls to approximately $1.3\%$ top-$1$ accuracy, while existing source-anchored CTTA methods continue applying the same anchor strength. We call this failure mode blind anchoring and propose RMemSafe, a reliability-gated extension of ROID that uses the frozen source's normalized predictive entropy to attenuate all explicit source-coupled uses in the objective. When the source posterior approaches uniformity, the gate closes: the source anchor and agreement filter vanish, and the objective reduces to a source-agnostic fallback comprising ROID's base losses plus marginal calibration. Combined with ASR, RMemSafe achieves the lowest error on $8$ of $9$ matched-split continual-corruption cells and is the best reset-based method on all $9$, improving ROID+ASR by $1.05$~pp on ResNet-50 and $0.48$~pp on ViT-B/16. A controlled source-degradation sweep shows a $1.13{\times}$ shallower harm slope than ROID+ASR, consistent with the graceful-decay prediction. The entropy gate detects high-entropy source collapse, not confidently wrong low-entropy sources; this scope is explicitly evaluated and discussed.