Distributionally robust risk evaluation with an isotonic constraint

TL;DR

This paper introduces a shape-constrained distributionally robust learning method that leverages isotonic constraints to improve robustness and accuracy under distribution shift, supported by theoretical and empirical evidence.

Contribution

It proposes a novel shape-constrained approach to distributionally robust learning using isotonic constraints, with theoretical guarantees and empirical validation.

Findings

Improved accuracy over traditional DRL methods.

Consistency of the empirical estimator under various settings.

Effective in synthetic and real data experiments.

Abstract

Statistical learning under distribution shift is challenging when neither prior knowledge nor fully accessible data from the target distribution is available. Distributionally robust learning (DRL) aims to control the worst-case statistical performance within an uncertainty set of candidate distributions, but how to properly specify the set remains challenging. To enable distributional robustness without being overly conservative, in this paper, we propose a shape-constrained approach to DRL, which incorporates prior information about the way in which the unknown target distribution differs from its estimate. More specifically, we assume the unknown density ratio between the target distribution and its estimate is isotonic with respect to some partial order. At the population level, we provide a solution to the shape-constrained optimization problem that does not involve the isotonic constraint. At the sample level, we provide consistency results for an empirical estimator of the target in a range of different settings. Empirical studies on both synthetic and real data examples demonstrate the improved accuracy of the proposed shape-constrained approach.