Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift

TL;DR

This paper introduces a minimax optimal two-stage algorithm for estimating moments under covariate shift, addressing distributional differences between training and testing data with theoretical guarantees and practical robustness.

Contribution

It proposes a novel two-stage method achieving minimax optimal bounds for moment estimation under covariate shift, including a truncated estimator for unknown distributions.

Findings

The proposed method attains minimax optimal bounds up to a logarithmic factor.

The truncated estimator ensures double robustness in practical scenarios.

Numerical experiments confirm the theoretical advantages of the approach.

Abstract

Covariate shift occurs when the distribution of input features differs between the training and testing phases. In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its common occurrence in real-world scenarios. In this paper, we investigate the minimax lower bound of the problem when the source and target distributions are known. To achieve the minimax optimal bound (up to a logarithmic factor), we propose a two-stage algorithm. Specifically, it first trains an optimal estimator for the function under the source distribution, and then uses a likelihood ratio reweighting procedure to calibrate the moment estimator. In practice, the source and target distributions are typically unknown, and estimating the likelihood ratio may be unstable. To solve this problem, we propose a truncated version of the estimator that ensures double robustness and provide the corresponding upper bound. Extensive numerical studies on synthetic examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method.