Gradual Domain Adaptation via Manifold-Constrained Distributionally Robust Optimization

TL;DR

This paper introduces a method for gradual domain adaptation using manifold constraints and distributionally robust optimization, providing theoretical error bounds and demonstrating effectiveness through experiments.

Contribution

It proposes a novel DRO-based approach with adaptive Wasserstein radius for gradual domain shifts, along with theoretical error bounds and a compatibility measure.

Findings

Error bounds depend on distribution constraints and can be linear or exponential.

The method guarantees classification error control across a sequence of shifting distributions.

Experimental results validate the theoretical guarantees and effectiveness of the approach.

Abstract

The aim of this paper is to address the challenge of gradual domain adaptation within a class of manifold-constrained data distributions. In particular, we consider a sequence of $T\ge2$ data distributions $P_1,\ldots,P_T$ undergoing a gradual shift, where each pair of consecutive measures $P_i,P_{i+1}$ are close to each other in Wasserstein distance. We have a supervised dataset of size $n$ sampled from $P_0$, while for the subsequent distributions in the sequence, only unlabeled i.i.d. samples are available. Moreover, we assume that all distributions exhibit a known favorable attribute, such as (but not limited to) having intra-class soft/hard margins. In this context, we propose a methodology rooted in Distributionally Robust Optimization (DRO) with an adaptive Wasserstein radius. We theoretically show that this method guarantees the classification error across all $P_i$s can be suitably bounded. Our bounds rely on a newly introduced {\it {compatibility}} measure, which fully characterizes the error propagation dynamics along the sequence. Specifically, for inadequately constrained distributions, the error can exponentially escalate as we progress through the gradual shifts. Conversely, for appropriately constrained distributions, the error can be demonstrated to be linear or even entirely eradicated. We have substantiated our theoretical findings through several experimental results.