A New PAC-Bayesian Perspective on Domain Adaptation

TL;DR

This paper introduces a novel PAC-Bayesian framework for domain adaptation, deriving bounds that relate source error and target disagreement, and proposes algorithms with experimental validation.

Contribution

It presents a new PAC-Bayesian upper-bound on target risk based on distribution divergence, leading to a specialized learning algorithm for domain adaptation.

Findings

Derived a new upper-bound on target risk involving distribution divergence.

Specialized the bound for linear classifiers.

Validated the approach with experiments on real data.

Abstract

We study the issue of PAC-Bayesian domain adaptation: We want to learn, from a source domain, a majority vote model dedicated to a target one. Our theoretical contribution brings a new perspective by deriving an upper-bound on the target risk where the distributions' divergence---expressed as a ratio---controls the trade-off between a source error measure and the target voters' disagreement. Our bound suggests that one has to focus on regions where the source data is informative.From this result, we derive a PAC-Bayesian generalization bound, and specialize it to linear classifiers. Then, we infer a learning algorithmand perform experiments on real data.