The Definitions of Interpretability and Learning of Interpretable Models

TL;DR

This paper introduces a formal mathematical definition of interpretability for machine learning models, proposes a framework for training fully human-interpretable models, and demonstrates their advantages in transparency and robustness through image dataset experiments.

Contribution

It provides a rigorous interpretability definition and a practical training framework for fully human-interpretable models, enhancing transparency and robustness.

Findings

Models offer human-understandable decision processes

Interpretable models show increased robustness against adversarial attacks

Framework effectively trains interpretable models using user interactions

Abstract

As machine learning algorithms getting adopted in an ever-increasing number of applications, interpretation has emerged as a crucial desideratum. In this paper, we propose a mathematical definition for the human-interpretable model. In particular, we define interpretability between two information process systems. If a prediction model is interpretable by a human recognition system based on the above interpretability definition, the prediction model is defined as a completely human-interpretable model. We further design a practical framework to train a completely human-interpretable model by user interactions. Experiments on image datasets show the advantages of our proposed model in two aspects: 1) The completely human-interpretable model can provide an entire decision-making process that is human-understandable; 2) The completely human-interpretable model is more robust against adversarial attacks.