Quantifying the Reliance of Black-Box Decision-Makers on Variables of Interest

TL;DR

This paper proposes a permutation-based framework to quantify how much black-box decision-makers depend on specific variables, with applications in legal decision analysis and policy implications.

Contribution

It introduces a novel, adaptable measure of reliance on variables of interest for black-box models, with theoretical, computational, and practical insights.

Findings

Supreme Court Justices' reliance on gender is less than previously estimated.

The framework enables comparison of reliance on different variables beyond regression coefficients.

Illustrative example demonstrates policy relevance of reliance measures.

Abstract

This paper introduces a framework for measuring how much black-box decision-makers rely on variables of interest. The framework adapts a permutation-based measure of variable importance from the explainable machine learning literature. With an emphasis on applicability, I present some of the framework's theoretical and computational properties, explain how reliance computations have policy implications, and work through an illustrative example. In the empirical application to interruptions by Supreme Court Justices during oral argument, I find that the effect of gender is more muted compared to the existing literature's estimate; I then use this paper's framework to compare Justices' reliance on gender and alignment to their reliance on experience, which are incomparable using regression coefficients.