Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values

TL;DR

GAM Changer is an interactive system that enables domain experts and data scientists to edit and align Generalized Additive Models with human knowledge and values, addressing interpretability issues in ML models.

Contribution

We introduce GAM Changer, the first tool allowing responsible, interactive editing of GAMs to reflect human insights, improving model transparency and trust.

Findings

Physicians used GAM Changer to investigate and fix risk models.

Data scientists found the tool easy to use and workflow-compatible.

The tool is accessible via web browsers and notebooks.

Abstract

Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collaboration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action--empowering users to analyze, validate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneumonia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users' web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the following public demo link: https://interpret.ml/gam-changer.