# TransAdapt: A Transformative Framework for Online Test Time Adaptive Semantic Segmentation

## TL;DR

TransAdapt introduces a transformer-based framework for online test-time adaptation in semantic segmentation, improving accuracy without online training by leveraging input transformations and invariance enforcement.

## Contribution

It proposes a novel transformer module for test-time adaptation that enhances segmentation quality without requiring online training, using input transformations and unsupervised loss functions.

## Key findings

- Up to 17.6% mIOU improvement over no-adaptation.
- Outperforms competitive baselines in online test-time adaptation.
- Effective in real-world, online test scenarios.

## Abstract

Test-time adaptive (TTA) semantic segmentation adapts a source pre-trained image semantic segmentation model to unlabeled batches of target domain test images, different from real-world, where samples arrive one-by-one in an online fashion. To tackle online settings, we propose TransAdapt, a framework that uses transformer and input transformations to improve segmentation performance. Specifically, we pre-train a transformer-based module on a segmentation network that transforms unsupervised segmentation output to a more reliable supervised output, without requiring test-time online training. To also facilitate test-time adaptation, we propose an unsupervised loss based on the transformed input that enforces the model to be invariant and equivariant to photometric and geometric perturbations, respectively. Overall, our framework produces higher quality segmentation masks with up to 17.6% and 2.8% mIOU improvement over no-adaptation and competitive baselines, respectively.