Calibration of Network Confidence for Unsupervised Domain Adaptation Using Estimated Accuracy

TL;DR

This paper proposes a novel calibration method for neural network confidence in unsupervised domain adaptation by estimating target domain accuracy, improving confidence calibration without labeled target data.

Contribution

Introduces a calibration procedure that estimates target accuracy to adjust confidence scores, outperforming importance weighting methods in unsupervised domain adaptation.

Findings

Significantly improves confidence calibration accuracy

Outperforms existing importance weighting methods

Effective across multiple standard datasets

Abstract

This study addresses the problem of calibrating network confidence while adapting a model that was originally trained on a source domain to a target domain using unlabeled samples from the target domain. The absence of labels from the target domain makes it impossible to directly calibrate the adapted network on the target domain. To tackle this challenge, we introduce a calibration procedure that relies on estimating the network's accuracy on the target domain. The network accuracy is first computed on the labeled source data and then is modified to represent the actual accuracy of the model on the target domain. The proposed algorithm calibrates the prediction confidence directly in the target domain by minimizing the disparity between the estimated accuracy and the computed confidence. The experimental results show that our method significantly outperforms existing methods, which rely on importance weighting, across several standard datasets.