Designing Inherently Interpretable Machine Learning Models

TL;DR

This paper discusses the design principles and practical approaches for creating inherently interpretable machine learning models, emphasizing transparency and regulatory compliance in high-stakes industries.

Contribution

It introduces a qualitative template for assessing interpretability and provides design principles with real-world examples for developing high-performance, inherently interpretable models.

Findings

Proposed a qualitative template for interpretability assessment

Reviewed recent works on interpretable models like ExNN, GAMI-Net, SIMTree

Demonstrated design of an interpretable ReLU DNN for credit default prediction

Abstract

Interpretable machine learning (IML) becomes increasingly important in highly regulated industry sectors related to the health and safety or fundamental rights of human beings. In general, the inherently IML models should be adopted because of their transparency and explainability, while black-box models with model-agnostic explainability can be more difficult to defend under regulatory scrutiny. For assessing inherent interpretability of a machine learning model, we propose a qualitative template based on feature effects and model architecture constraints. It provides the design principles for high-performance IML model development, with examples given by reviewing our recent works on ExNN, GAMI-Net, SIMTree, and the Aletheia toolkit for local linear interpretability of deep ReLU networks. We further demonstrate how to design an interpretable ReLU DNN model with evaluation of conceptual soundness for a real case study of predicting credit default in home lending. We hope that this work will provide a practical guide of developing inherently IML models in high risk applications in banking industry, as well as other sectors.