Safe Distributionally Robust Feature Selection under Covariate Shift

TL;DR

This paper introduces safe-DRFS, a novel feature selection method for distributionally robust learning under covariate shift, ensuring reliable sensor subset selection with theoretical guarantees in diverse deployment environments.

Contribution

It extends safe screening techniques to distributionally robust feature selection, providing a method with finite-sample guarantees to identify all potentially optimal feature subsets under covariate shift.

Findings

Proposes safe-DRFS with finite-sample guarantees

Ensures inclusion of all potentially optimal features under distribution shifts

Addresses practical multi-sensor system deployment challenges

Abstract

In practical machine learning, the environments encountered during the model development and deployment phases often differ, especially when a model is used by many users in diverse settings. Learning models that maintain reliable performance across plausible deployment environments is known as distributionally robust (DR) learning. In this work, we study the problem of distributionally robust feature selection (DRFS), with a particular focus on sparse sensing applications motivated by industrial needs. In practical multi-sensor systems, a shared subset of sensors is typically selected prior to deployment based on performance evaluations using many available sensors. At deployment, individual users may further adapt or fine-tune models to their specific environments. When deployment environments differ from those anticipated during development, this strategy can result in systems lacking sensors required for optimal performance. To address this issue, we propose safe-DRFS, a novel approach that extends safe screening from conventional sparse modeling settings to a DR setting under covariate shift. Our method identifies a feature subset that encompasses all subsets that may become optimal across a specified range of input distribution shifts, with finite-sample theoretical guarantees of no false feature elimination.