Predicting Class Distribution Shift for Reliable Domain Adaptive Object Detection

TL;DR

This paper introduces a novel framework for addressing class distribution shifts in unsupervised domain adaptive object detection, improving pseudo-label reliability and overall detection performance in open-world environments.

Contribution

It proposes using a pre-trained vision-language model to predict class distributions and align pseudo-labels accordingly, enhancing self-training robustness.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Achieves a 4.7 mAP improvement under challenging class distribution shifts.

Effectively improves pseudo-label accuracy and detection reliability.

Abstract

Unsupervised Domain Adaptive Object Detection (UDA-OD) uses unlabelled data to improve the reliability of robotic vision systems in open-world environments. Previous approaches to UDA-OD based on self-training have been effective in overcoming changes in the general appearance of images. However, shifts in a robot's deployment environment can also impact the likelihood that different objects will occur, termed class distribution shift. Motivated by this, we propose a framework for explicitly addressing class distribution shift to improve pseudo-label reliability in self-training. Our approach uses the domain invariance and contextual understanding of a pre-trained joint vision and language model to predict the class distribution of unlabelled data. By aligning the class distribution of pseudo-labels with this prediction, we provide weak supervision of pseudo-label accuracy. To further account for low quality pseudo-labels early in self-training, we propose an approach to dynamically adjust the number of pseudo-labels per image based on model confidence. Our method outperforms state-of-the-art approaches on several benchmarks, including a 4.7 mAP improvement when facing challenging class distribution shift.