Efficient Learning with Arbitrary Covariate Shift

TL;DR

This paper introduces a polynomial-time algorithm for covariate shift learning that leverages reliable learning oracles, providing an optimal reduction between reliable and PQ learning in the context of semi-supervised and selective classification.

Contribution

It presents the first efficient PQ-learning algorithm that reduces to reliable learning, establishing the equivalence between these two models.

Findings

Provides a polynomial-time PQ-learning algorithm using reliable oracles.

Shows the equivalence of reliable and PQ learning models.

Achieves optimal reduction in covariate shift scenarios.

Abstract

We give an efficient algorithm for learning a binary function in a given class C of bounded VC dimension, with training data distributed according to P and test data according to Q, where P and Q may be arbitrary distributions over X. This is the generic form of what is called covariate shift, which is impossible in general as arbitrary P and Q may not even overlap. However, recently guarantees were given in a model called PQ-learning (Goldwasser et al., 2020) where the learner has: (a) access to unlabeled test examples from Q (in addition to labeled samples from P, i.e., semi-supervised learning); and (b) the option to reject any example and abstain from classifying it (i.e., selective classification). The algorithm of Goldwasser et al. (2020) requires an (agnostic) noise tolerant learner for C. The present work gives a polynomial-time PQ-learning algorithm that uses an oracle to a "reliable" learner for C, where reliable learning (Kalai et al., 2012) is a model of learning with one-sided noise. Furthermore, our reduction is optimal in the sense that we show the equivalence of reliable and PQ learning.