On $f$-Divergence Principled Domain Adaptation: An Improved Framework

TL;DR

This paper advances the theoretical understanding of unsupervised domain adaptation by refining $f$-divergence measures, introducing $f$-domain discrepancy, and demonstrating improved empirical performance on benchmarks.

Contribution

It introduces a new $f$-domain discrepancy measure, refines theoretical bounds, and bridges the gap between algorithms and theory in unsupervised domain adaptation.

Findings

$f$-DD improves target error bounds.

Enhanced sample complexity bounds are derived.

Empirical results show superior performance on benchmarks.

Abstract

Unsupervised domain adaptation (UDA) plays a crucial role in addressing distribution shifts in machine learning. In this work, we improve the theoretical foundations of UDA proposed in Acuna et al. (2021) by refining their $f$-divergence-based discrepancy and additionally introducing a new measure, $f$-domain discrepancy ($f$-DD). By removing the absolute value function and incorporating a scaling parameter, $f$-DD obtains novel target error and sample complexity bounds, allowing us to recover previous KL-based results and bridging the gap between algorithms and theory presented in Acuna et al. (2021). Using a localization technique, we also develop a fast-rate generalization bound. Empirical results demonstrate the superior performance of $f$-DD-based learning algorithms over previous works in popular UDA benchmarks.