Talking About the Assumption in the Room

TL;DR

This paper explores how ML practitioners conceptualize and manage assumptions in their workflows, revealing confusions and proposing recommendations to improve understanding and handling of assumptions in ML development.

Contribution

It introduces a new perspective using Informal Logic to clarify assumptions and provides empirical insights from interviews with ML practitioners about their assumptions management.

Findings

Assumptions are constructed independently by practitioners.

Practitioners handle assumptions reactively and reflectively.

Assumptions are recorded nebulously and inconsistently.

Abstract

The reference to assumptions in how practitioners use or interact with machine learning (ML) systems is ubiquitous in HCI and responsible ML discourse. However, what remains unclear from prior works is the conceptualization of assumptions and how practitioners identify and handle assumptions throughout their workflows. This leads to confusion about what assumptions are and what needs to be done with them. We use the concept of an argument from Informal Logic, a branch of Philosophy, to offer a new perspective to understand and explicate the confusions surrounding assumptions. Through semi-structured interviews with 22 ML practitioners, we find what contributes most to these confusions is how independently assumptions are constructed, how reactively and reflectively they are handled, and how nebulously they are recorded. Our study brings the peripheral discussion of assumptions in ML to the center and presents recommendations for practitioners to better think about and work with assumptions.