The Doctor Just Won't Accept That!

TL;DR

This paper questions the assumption that stakeholders universally reject machine learning models and emphasizes the need to understand stakeholder preferences and feasibility for advancing interpretable AI.

Contribution

It critically examines claims about stakeholder rejection of machine learning and advocates for a nuanced understanding of stakeholder needs and realistic goals in interpretability.

Findings

Stakeholders' acceptance of models varies and depends on context.

Understanding stakeholder desiderata is crucial for progress in interpretability.

Feasibility of interpretability goals must be carefully evaluated.

Abstract

Calls to arms to build interpretable models express a well-founded discomfort with machine learning. Should a software agent that does not even know what a loan is decide who qualifies for one? Indeed, we ought to be cautious about injecting machine learning (or anything else, for that matter) into applications where there may be a significant risk of causing social harm. However, claims that stakeholders "just won't accept that!" do not provide a sufficient foundation for a proposed field of study. For the field of interpretable machine learning to advance, we must ask the following questions: What precisely won't various stakeholders accept? What do they want? Are these desiderata reasonable? Are they feasible? In order to answer these questions, we'll have to give real-world problems and their respective stakeholders greater consideration.