Sequential Domain Adaptation by Synthesizing Distributionally Robust Experts

TL;DR

This paper introduces a method to synthesize distributionally robust least squares estimators for supervised domain adaptation, improving predictive accuracy on target domain data by leveraging convex optimization and online aggregation.

Contribution

It proposes a novel approach to create a family of robust estimators using divergence-based moment conditions and applies online aggregation for sequential predictions.

Findings

Robust estimators outperform non-robust methods on real data.

Efficient convex optimization algorithms are used for estimator synthesis.

Sequential predictions benefit from the proposed robust strategies.

Abstract

Least squares estimators, when trained on a few target domain samples, may predict poorly. Supervised domain adaptation aims to improve the predictive accuracy by exploiting additional labeled training samples from a source distribution that is close to the target distribution. Given available data, we investigate novel strategies to synthesize a family of least squares estimator experts that are robust with regard to moment conditions. When these moment conditions are specified using Kullback-Leibler or Wasserstein-type divergences, we can find the robust estimators efficiently using convex optimization. We use the Bernstein online aggregation algorithm on the proposed family of robust experts to generate predictions for the sequential stream of target test samples. Numerical experiments on real data show that the robust strategies may outperform non-robust interpolations of the empirical least squares estimators.