Class Prior Estimation under Covariate Shift: No Problem?

TL;DR

This paper investigates the challenges of estimating class priors under covariate shift, showing that information loss in covariates can break the shift property, and proposes a probing algorithm as an alternative solution.

Contribution

It demonstrates that covariate shift may be lost due to information reduction and introduces a probing algorithm for class prior estimation under such conditions.

Findings

Covariate shift property can be lost when covariates are reduced.

Simple class prior estimation methods become infeasible under covariate shift.

A new probing algorithm is proposed for class prior estimation.

Abstract

We show that in the context of classification the property of source and target distributions to be related by covariate shift may be lost if the information content captured in the covariates is reduced, for instance by dropping components or mapping into a lower-dimensional or finite space. As a consequence, under covariate shift simple approaches to class prior estimation in the style of classify and count with or without adjustment are infeasible. We prove that transformations of the covariates that preserve the covariate shift property are necessarily sufficient in the statistical sense for the full set of covariates. A probing algorithm as alternative approach to class prior estimation under covariate shift is proposed.