Outcome Indistinguishability

TL;DR

This paper introduces Outcome Indistinguishability, a new concept inspired by cryptography, to evaluate the robustness of prediction algorithms against efficient refutation, with implications for algorithm auditing and fairness.

Contribution

It formalizes Outcome Indistinguishability, explores its hierarchy, constructs examples at all levels, and proves complexity lower bounds, advancing understanding of predictive model indistinguishability.

Findings

Outcome Indistinguishability behaves differently than previous notions.

Constructs exist at all hierarchy levels of Outcome Indistinguishability.

Hardness results suggest auditors need oracle access to predictors.

Abstract

Prediction algorithms assign numbers to individuals that are popularly understood as individual "probabilities" -- what is the probability of 5-year survival after cancer diagnosis? -- and which increasingly form the basis for life-altering decisions. Drawing on an understanding of computational indistinguishability developed in complexity theory and cryptography, we introduce Outcome Indistinguishability. Predictors that are Outcome Indistinguishable yield a generative model for outcomes that cannot be efficiently refuted on the basis of the real-life observations produced by Nature. We investigate a hierarchy of Outcome Indistinguishability definitions, whose stringency increases with the degree to which distinguishers may access the predictor in question. Our findings reveal that Outcome Indistinguishability behaves qualitatively differently than previously studied notions of indistinguishability. First, we provide constructions at all levels of the hierarchy. Then, leveraging recently-developed machinery for proving average-case fine-grained hardness, we obtain lower bounds on the complexity of the more stringent forms of Outcome Indistinguishability. This hardness result provides the first scientific grounds for the political argument that, when inspecting algorithmic risk prediction instruments, auditors should be granted oracle access to the algorithm, not simply historical predictions.