Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse

TL;DR

This paper introduces feature responsiveness scores as a model-agnostic way to improve explanations for high-stakes decisions, ensuring they genuinely support recourse by focusing on actionable features.

Contribution

It proposes a novel responsiveness metric and efficient computation methods, addressing limitations of existing feature importance explanations like SHAP and LIME.

Findings

Standard explanations can highlight unresponsive features, undermining recourse.

Responsiveness scores identify features that decision subjects can realistically change.

Applying these scores improves the effectiveness of explanations in lending scenarios.

Abstract

Consumer protection rules require companies that deploy models to automate decisions in high-stakes settings to explain predictions to decision subjects. These rules are motivated, in part, by the belief that explanations can promote recourse by revealing information that decision subjects can use to contest or overturn their predictions. In practice, companies provide individuals with a list of principal reasons based on feature importance derived from methods like SHAP and LIME. In this work, we show how common practices can fail to provide recourse and propose to highlight features based on their responsiveness -- the probability that a decision subject can attain a target prediction through an arbitrary intervention on the feature. We develop efficient methods to compute responsiveness scores for any model and actionability constraints. We show that standard practices in lending can undermine decision subjects by highlighting unresponsive features and explaining predictions that are fixed.