Inference on Consensus Ranking of Distributions

TL;DR

This paper develops statistical inference methods to determine the degree of consensus between two distributions based on their expected utilities, providing confidence sets and tests that account for broad utility function classes.

Contribution

It introduces a novel framework for inference on consensus rankings of distributions using confidence sets and multiple testing, justified by empirical process theory.

Findings

Confidence sets reliably contain the true set of utility functions

Method controls familywise error rate in multiple testing

Illustrated with simulated and real data examples

Abstract

Instead of testing for unanimous agreement, I propose learning how broad of a consensus favors one distribution over another (of earnings, productivity, asset returns, test scores, etc.). Specifically, given a sample from each of two distributions, I propose statistical inference methods to learn about the set of utility functions for which the first distribution has higher expected utility than the second distribution. With high probability, an "inner" confidence set is contained within this true set, while an "outer" confidence set contains the true set. Such confidence sets can be formed by inverting a proposed multiple testing procedure that controls the familywise error rate. Theoretical justification comes from empirical process results, given that very large classes of utility functions are generally Donsker (subject to finite moments). The theory additionally justifies a uniform (over utility functions) confidence band of expected utility differences, as well as tests with a utility-based "restricted stochastic dominance" as either the null or alternative hypothesis. Simulated and empirical examples illustrate the methodology.