Machine Predictions and Human Decisions with Variation in Payoffs and Skill

TL;DR

This paper develops a framework integrating machine learning predictions with heterogeneous human decision-making, applied to antibiotic prescribing, revealing significant potential for reducing unnecessary prescriptions and improving health outcomes.

Contribution

It introduces a novel framework that models decision heterogeneity and applies it to health policy, demonstrating how machine learning can enhance decision quality in complex settings.

Findings

Machine learning predictions combined with physician skill reduce antibiotic prescribing by 25.4%.

Significant variation exists in physicians' diagnostic skill and decision trade-offs.

Policy simulations show welfare gains from integrating machine learning with human decision-making.

Abstract

Human decision-making differs due to variation in both incentives and available information. This constitutes a substantial challenge for the evaluation of whether and how machine learning predictions can improve decision outcomes. We propose a framework that incorporates machine learning on large-scale data into a choice model featuring heterogeneity in decision maker payoff functions and predictive skill. We apply this framework to the major health policy problem of improving the efficiency in antibiotic prescribing in primary care, one of the leading causes of antibiotic resistance. Our analysis reveals large variation in physicians' skill to diagnose bacterial infections and in how physicians trade off the externality inherent in antibiotic use against its curative benefit. Counterfactual policy simulations show that the combination of machine learning predictions with physician diagnostic skill results in a 25.4 percent reduction in prescribing and achieves the largest welfare gains compared to alternative policies for both estimated physician as well as conservative social planner preference weights on the antibiotic resistance externality.