Introducing Intermediate Domains for Effective Self-Training during Test-Time

TL;DR

This paper proposes GTTA, a method that introduces artificial intermediate domains using mixup and style transfer to improve self-training during test-time adaptation, especially under non-gradual domain shifts.

Contribution

The paper introduces GTTA, a novel approach that creates artificial intermediate domains to enhance self-training effectiveness in test-time adaptation under various domain shift scenarios.

Findings

GTTA improves performance on continual and gradual corruption benchmarks.

GTTA outperforms existing methods in urban scene segmentation under non-stationary shifts.

The new CarlaTTA benchmark facilitates research on non-stationary domain shifts.

Abstract

Experiencing domain shifts during test-time is nearly inevitable in practice and likely results in a severe performance degradation. To overcome this issue, test-time adaptation continues to update the initial source model during deployment. A promising direction are methods based on self-training which have been shown to be well suited for gradual domain adaptation, since reliable pseudo-labels can be provided. In this work, we address two problems that exist when applying self-training in the setting of test-time adaptation. First, adapting a model to long test sequences that contain multiple domains can lead to error accumulation. Second, naturally, not all shifts are gradual in practice. To tackle these challenges, we introduce GTTA. By creating artificial intermediate domains that divide the current domain shift into a more gradual one, effective self-training through high quality pseudo-labels can be performed. To create the intermediate domains, we propose two independent variations: mixup and light-weight style transfer. We demonstrate the effectiveness of our approach on the continual and gradual corruption benchmarks, as well as ImageNet-R. To further investigate gradual shifts in the context of urban scene segmentation, we publish a new benchmark: CarlaTTA. It enables the exploration of several non-stationary domain shifts.