Domain Adaptation with Cauchy-Schwarz Divergence

TL;DR

This paper introduces the Cauchy-Schwarz divergence for unsupervised domain adaptation, providing a tighter theoretical bound and a practical estimator for distribution discrepancy, enhancing adaptation performance.

Contribution

It presents the CS divergence as a new measure for domain discrepancy, with a simple estimator applicable without distributional assumptions, improving UDA methods.

Findings

CS divergence offers a tighter generalization bound than KL divergence.

The estimator effectively measures distribution discrepancy in representation space.

Application in various UDA frameworks improves adaptation performance.

Abstract

Domain adaptation aims to use training data from one or multiple source domains to learn a hypothesis that can be generalized to a different, but related, target domain. As such, having a reliable measure for evaluating the discrepancy of both marginal and conditional distributions is crucial. We introduce Cauchy-Schwarz (CS) divergence to the problem of unsupervised domain adaptation (UDA). The CS divergence offers a theoretically tighter generalization error bound than the popular Kullback-Leibler divergence. This holds for the general case of supervised learning, including multi-class classification and regression. Furthermore, we illustrate that the CS divergence enables a simple estimator on the discrepancy of both marginal and conditional distributions between source and target domains in the representation space, without requiring any distributional assumptions. We provide multiple examples to illustrate how the CS divergence can be conveniently used in both distance metric- or adversarial training-based UDA frameworks, resulting in compelling performance.