The Relative Value of Prediction in Algorithmic Decision Making

TL;DR

This paper explores the importance of prediction in algorithmic decision making, comparing its welfare benefits to other policy tools through theoretical analysis and practical implications.

Contribution

It provides a formal framework and sharp conditions to assess the relative value of prediction versus other policy levers in social decision systems.

Findings

Theoretical conditions for the value of prediction in welfare improvement.

Comparison of prediction benefits to expanding access in statistical models.

Guidelines for designing decision systems based on theoretical insights.

Abstract

Algorithmic predictions are increasingly used to inform the allocations of goods and interventions in the public sphere. In these domains, predictions serve as a means to an end. They provide stakeholders with insights into likelihood of future events as a means to improve decision making quality, and enhance social welfare. However, if maximizing welfare is the ultimate goal, prediction is only a small piece of the puzzle. There are various other policy levers a social planner might pursue in order to improve bottom-line outcomes, such as expanding access to available goods, or increasing the effect sizes of interventions. Given this broad range of design decisions, a basic question to ask is: What is the relative value of prediction in algorithmic decision making? How do the improvements in welfare arising from better predictions compare to those of other policy levers? The goal of our work is to initiate the formal study of these questions. Our main results are theoretical in nature. We identify simple, sharp conditions determining the relative value of prediction vis-\`a-vis expanding access, within several statistical models that are popular amongst quantitative social scientists. Furthermore, we illustrate how these theoretical insights may be used to guide the design of algorithmic decision making systems in practice.