Constraining Pseudo-label in Self-training Unsupervised Domain Adaptation with Energy-based Model

TL;DR

This paper introduces an energy-based model to improve pseudo-label reliability in self-training for unsupervised domain adaptation, enhancing performance on image classification and segmentation tasks.

Contribution

It proposes a novel energy-based regularization framework for self-training in UDA, addressing pseudo-label unreliability and maintaining discriminative accuracy.

Findings

Improved accuracy on UDA benchmarks for image classification.

Effective pseudo-label constraint via energy minimization.

Demonstrated generality across classification and segmentation tasks.

Abstract

Deep learning is usually data starved, and the unsupervised domain adaptation (UDA) is developed to introduce the knowledge in the labeled source domain to the unlabeled target domain. Recently, deep self-training presents a powerful means for UDA, involving an iterative process of predicting the target domain and then taking the confident predictions as hard pseudo-labels for retraining. However, the pseudo-labels are usually unreliable, thus easily leading to deviated solutions with propagated errors. In this paper, we resort to the energy-based model and constrain the training of the unlabeled target sample with an energy function minimization objective. It can be achieved via a simple additional regularization or an energy-based loss. This framework allows us to gain the benefits of the energy-based model, while retaining strong discriminative performance following a plug-and-play fashion. The convergence property and its connection with classification expectation minimization are investigated. We deliver extensive experiments on the most popular and large-scale UDA benchmarks of image classification as well as semantic segmentation to demonstrate its generality and effectiveness.