Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputs

TL;DR

This paper introduces visual analytics tools that help users intuitively assess machine learning model reliability by exploring input neighbors and editing inputs, demonstrated through a medical case study.

Contribution

It presents novel interactive visualization modules that improve understanding of model uncertainty and reliability without complex visualizations or deep ML expertise.

Findings

Physicians better aligned model uncertainty with domain factors.

Enhanced intuition about model capabilities and limitations.

Improved interpretability over baseline methods.

Abstract

Interpretability methods aim to help users build trust in and understand the capabilities of machine learning models. However, existing approaches often rely on abstract, complex visualizations that poorly map to the task at hand or require non-trivial ML expertise to interpret. Here, we present two visual analytics modules that facilitate an intuitive assessment of model reliability. To help users better characterize and reason about a model's uncertainty, we visualize raw and aggregate information about a given input's nearest neighbors. Using an interactive editor, users can manipulate this input in semantically-meaningful ways, determine the effect on the output, and compare against their prior expectations. We evaluate our interface using an electrocardiogram beat classification case study. Compared to a baseline feature importance interface, we find that 14 physicians are better able to align the model's uncertainty with domain-relevant factors and build intuition about its capabilities and limitations.