Unpacking the Black Box: Regulating Algorithmic Decisions

TL;DR

This paper models regulatory oversight of complex algorithms in high-stakes settings, showing that targeted regulation of specific issues can outperform broad transparency requirements, with empirical evidence from consumer lending.

Contribution

It introduces a theoretical framework for regulating black-box algorithms, highlighting the inefficiency of full transparency and advocating for targeted regulation based on specific misalignments.

Findings

Complex models can outperform simple transparent ones when regulation is targeted.

Targeted regulation focusing on specific issues yields Pareto improvements.

Empirical analysis in lending supports the benefits of complex, regulated models.

Abstract

What should regulators of complex algorithms regulate? We propose a model of oversight over 'black-box' algorithms used in high-stakes applications such as lending, medical testing, or hiring. In our model, a regulator is limited in how much she can learn about a black-box model deployed by an agent with misaligned preferences. The regulator faces two choices: first, whether to allow for the use of complex algorithms; and second, which key properties of algorithms to regulate. We show that limiting agents to algorithms that are simple enough to be fully transparent is inefficient as long as the misalignment is limited and complex algorithms have sufficiently better performance than simple ones. Allowing for complex algorithms can improve welfare, but the gains depend on how the regulator regulates them. Regulation that focuses on the overall average behavior of algorithms, for example based on standard explainer tools, will generally be inefficient. Targeted regulation that focuses on the source of incentive misalignment, e.g., excess false positives or racial disparities, can provide second-best solutions. We provide empirical support for our theoretical findings using an application in consumer lending, where we document that complex models regulated based on context-specific explanation tools outperform simple, fully transparent models. This gain from complex models represents a Pareto improvement across our empirical applications that is preferred both by the lender and from the perspective of the financial regulator.