Synchronizing Task Behavior: Aligning Multiple Tasks during Test-Time Training

TL;DR

This paper introduces S4T, a novel test-time training method that synchronizes multiple task behaviors during domain shifts, improving performance on multi-task benchmarks.

Contribution

S4T is the first approach to explicitly synchronize multiple tasks during test-time training, addressing a key limitation of conventional methods.

Findings

S4T outperforms existing TTT methods on multiple benchmarks.

Predicting task relations is crucial for task synchronization.

S4T effectively handles multiple tasks under domain shifts.

Abstract

Generalizing neural networks to unseen target domains is a significant challenge in real-world deployments. Test-time training (TTT) addresses this by using an auxiliary self-supervised task to reduce the domain gap caused by distribution shifts between the source and target. However, we find that when models are required to perform multiple tasks under domain shifts, conventional TTT methods suffer from unsynchronized task behavior, where the adaptation steps needed for optimal performance in one task may not align with the requirements of other tasks. To address this, we propose a novel TTT approach called Synchronizing Tasks for Test-time Training (S4T), which enables the concurrent handling of multiple tasks. The core idea behind S4T is that predicting task relations across domain shifts is key to synchronizing tasks during test time. To validate our approach, we apply S4T to conventional multi-task benchmarks, integrating it with traditional TTT protocols. Our empirical results show that S4T outperforms state-of-the-art TTT methods across various benchmarks.