Distributionally Robust Coreset Selection under Covariate Shift

TL;DR

This paper introduces Distributionally Robust Coreset Selection (DRCS), a method for selecting training data that remains effective under unknown distribution shifts, especially covariate shift, improving robustness in deployment.

Contribution

The paper proposes DRCS, a novel coreset selection method that minimizes an upper bound on worst-case test error under distributional shifts, applicable to both convex and deep learning models.

Findings

DRCS effectively reduces worst-case test error under covariate shift.

Experimental results show improved robustness compared to traditional coreset methods.

The approach is adaptable to deep learning with suitable approximations.

Abstract

Coreset selection, which involves selecting a small subset from an existing training dataset, is an approach to reducing training data, and various approaches have been proposed for this method. In practical situations where these methods are employed, it is often the case that the data distributions differ between the development phase and the deployment phase, with the latter being unknown. Thus, it is challenging to select an effective subset of training data that performs well across all deployment scenarios. We therefore propose Distributionally Robust Coreset Selection (DRCS). DRCS theoretically derives an estimate of the upper bound for the worst-case test error, assuming that the future covariate distribution may deviate within a defined range from the training distribution. Furthermore, by selecting instances in a way that suppresses the estimate of the upper bound for the worst-case test error, DRCS achieves distributionally robust training instance selection. This study is primarily applicable to convex training computation, but we demonstrate that it can also be applied to deep learning under appropriate approximations. In this paper, we focus on covariate shift, a type of data distribution shift, and demonstrate the effectiveness of DRCS through experiments.