Balance-Subsampled Stable Prediction

TL;DR

This paper introduces the BSSP algorithm, which uses fractional factorial design to mitigate distribution shift effects and enhance prediction stability in machine learning models under sample selection bias.

Contribution

The paper proposes a novel BSSP algorithm that reduces confounding among predictors caused by distribution shift, improving stability and accuracy of predictions.

Findings

BSSP outperforms baseline methods in stability across test data.

The method effectively isolates predictor effects under distribution shift.

Numerical experiments validate the approach on synthetic and real data.

Abstract

In machine learning, it is commonly assumed that training and test data share the same population distribution. However, this assumption is often violated in practice because the sample selection bias may induce the distribution shift from training data to test data. Such a model-agnostic distribution shift usually leads to prediction instability across unknown test data. In this paper, we propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. It isolates the clear effect of each predictor from the confounding variables. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift, hence improve both the accuracy of parameter estimation and prediction stability. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.