Mixup for Test-Time Training

TL;DR

This paper introduces MixTTT, a mixup-based regularization technique for test-time training that mitigates overfitting and mismatch issues, improving domain adaptation performance.

Contribution

It proposes a novel mixup-based regularization method for test-time training, addressing overfitting and mismatch problems, and providing theoretical analysis of its benefits.

Findings

MixTTT improves test-time training performance across multiple benchmarks.

Theoretical analysis shows MixTTT alleviates mismatch between model parts.

Experimental results confirm effectiveness of MixTTT as an add-on module.

Abstract

Test-time training provides a new approach solving the problem of domain shift. In its framework, a test-time training phase is inserted between training phase and test phase. During test-time training phase, usually parts of the model are updated with test sample(s). Then the updated model will be used in the test phase. However, utilizing test samples for test-time training has some limitations. Firstly, it will lead to overfitting to the test-time procedure thus hurt the performance on the main task. Besides, updating part of the model without changing other parts will induce a mismatch problem. Thus it is hard to perform better on the main task. To relieve above problems, we propose to use mixup in test-time training (MixTTT) which controls the change of model's parameters as well as completing the test-time procedure. We theoretically show its contribution in alleviating the mismatch problem of updated part and static part for the main task as a specific regularization effect for test-time training. MixTTT can be used as an add-on module in general test-time training based methods to further improve their performance. Experimental results show the effectiveness of our method.