TITAN: Query-Token based Domain Adaptive Adversarial Learning

TL;DR

TITAN introduces a novel domain adaptive object detection method that improves pseudo-label reliability by separating target images based on similarity to source and employing adversarial modules, leading to significant performance gains.

Contribution

The paper proposes a Target-based Iterative Query-Token Adversarial Network (TITAN) that enhances source-free domain adaptation by partitioning target data and reducing domain gaps with adversarial learning.

Findings

Achieved up to 22.7% mAP improvement on several benchmarks.

Effectively separates target images into easy and hard subsets.

Demonstrated superior performance on natural and medical datasets.

Abstract

We focus on the source-free domain adaptive object detection (SF-DAOD) problem when source data is unavailable during adaptation and the model must adapt to an unlabeled target domain. The majority of approaches for the problem employ a self-supervised approach using a student-teacher (ST) framework where pseudo-labels are generated via a source-pretrained model for further fine-tuning. We observe that the performance of a student model often degrades drastically, due to the collapse of the teacher model, primarily caused by high noise in pseudo-labels, resulting from domain bias, discrepancies, and a significant domain shift across domains. To obtain reliable pseudo-labels, we propose a Target-based Iterative Query-Token Adversarial Network (TITAN), which separates the target images into two subsets: those similar to the source (easy) and those dissimilar (hard). We propose a strategy to estimate variance to partition the target domain. This approach leverages the insight that higher detection variances correspond to higher recall and greater similarity to the source domain. Also, we incorporate query-token-based adversarial modules into a student-teacher baseline framework to reduce the domain gaps between two feature representations. Experiments conducted on four natural imaging datasets and two challenging medical datasets have substantiated the superior performance of TITAN compared to existing state-of-the-art (SOTA) methodologies. We report an mAP improvement of +22.7, +22.2, +21.1, and +3.7 percent over the current SOTA on C2F, C2B, S2C, and K2C benchmarks, respectively.