Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory

TL;DR

This paper introduces a novel interpretability framework for AI based on sensemaking theory, emphasizing understanding the human audience's needs and context to improve explanations of ML models.

Contribution

It proposes a sensemaking-based approach to AI interpretability, focusing on stakeholder-specific explanations and design guidelines for more human-centric AI explanations.

Findings

Defines properties influencing human understanding, such as identity and social context.

Provides design guidelines for Sensible AI to enhance interpretability.

Connects organizational sensemaking principles to AI explanation design.

Abstract

Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact -- an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick's sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders' needs -- we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for Sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.