Distributionally Robust Safe Screening

TL;DR

This paper introduces a novel method called Distributionally Robust Safe Screening (DRSS) that improves model robustness by efficiently identifying irrelevant data points and features under distributional shifts, validated through theoretical guarantees and experiments.

Contribution

The paper develops DRSS, integrating distributionally robust learning with safe screening, to reliably exclude unnecessary samples and features under distributional uncertainty.

Findings

The DRSS method effectively identifies irrelevant samples and features.

Theoretical guarantees support the reliability of DRSS.

Numerical experiments demonstrate improved robustness and efficiency.

Abstract

In this study, we propose a method Distributionally Robust Safe Screening (DRSS), for identifying unnecessary samples and features within a DR covariate shift setting. This method effectively combines DR learning, a paradigm aimed at enhancing model robustness against variations in data distribution, with safe screening (SS), a sparse optimization technique designed to identify irrelevant samples and features prior to model training. The core concept of the DRSS method involves reformulating the DR covariate-shift problem as a weighted empirical risk minimization problem, where the weights are subject to uncertainty within a predetermined range. By extending the SS technique to accommodate this weight uncertainty, the DRSS method is capable of reliably identifying unnecessary samples and features under any future distribution within a specified range. We provide a theoretical guarantee of the DRSS method and validate its performance through numerical experiments on both synthetic and real-world datasets.