Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurement

TL;DR

This paper introduces a new statistical comparison method for random variables in spaces with locally varying measurement scales, addressing a gap in properly exploiting such complex data structures.

Contribution

It proposes a generalized stochastic dominance order, along with a regularized statistical test, operationalized via linear optimization and robustified with imprecise probabilities.

Findings

Test applied to multidimensional poverty data

Method demonstrated in finance and medicine datasets

Robustness improved with imprecise probability models

Abstract

Spaces with locally varying scale of measurement, like multidimensional structures with differently scaled dimensions, are pretty common in statistics and machine learning. Nevertheless, it is still understood as an open question how to exploit the entire information encoded in them properly. We address this problem by considering an order based on (sets of) expectations of random variables mapping into such non-standard spaces. This order contains stochastic dominance and expectation order as extreme cases when no, or respectively perfect, cardinal structure is given. We derive a (regularized) statistical test for our proposed generalized stochastic dominance (GSD) order, operationalize it by linear optimization, and robustify it by imprecise probability models. Our findings are illustrated with data from multidimensional poverty measurement, finance, and medicine.